2024-02-14T17:24:46.288 INFO:root:teuthology version: 0.0.1.dev258+g9e44c16 2024-02-14T17:24:46.288 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} --owner scheduled_gabrioux@teuthology --archive /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793 --name gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi -- /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793/orig.config.yaml 2024-02-14T17:24:46.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T17:24:46.393 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793 branch: wip-guits-testing-2024-02-14-1033 description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7559793' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-testing-2024-02-14-1033 ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) mon_bind_msgr2: false sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-testing-2024-02-14-1033 sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4718 sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-testing-2024-02-14-1033 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8727ce939fc044a98f215e29696bb1433f5758d1/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 targets: smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMsvxKdQsDa8qMY7n8wMF12iX6/2AeZvMJA+NH0/z+tBOwWZPHhgYwtuPScajjHIMpOmxa7cWAqHdYjT3yJ+Me0= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCWf6xgbepP4XQoSuLV9ftXptjCbEYXLJNvnDl37roFCrxZjL69Fc0q1OiLtzlR9cNyA6aDkgHYTXjOPx4mymw= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9e44c16382dfc4c612f7437cbb344c76223137cc timestamp: 2024-02-14_13:54:10 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-14T17:24:46.394 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8727ce939fc044a98f215e29696bb1433f5758d1/qa; will attempt to use it 2024-02-14T17:24:46.394 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8727ce939fc044a98f215e29696bb1433f5758d1/qa/tasks 2024-02-14T17:24:46.395 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-14T17:24:46.396 INFO:teuthology.task.internal:Checking packages... 2024-02-14T17:24:46.422 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8727ce939fc044a98f215e29696bb1433f5758d1' 2024-02-14T17:24:46.422 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-14T17:24:46.423 INFO:teuthology.packaging:ref: None 2024-02-14T17:24:46.423 INFO:teuthology.packaging:tag: None 2024-02-14T17:24:46.423 INFO:teuthology.packaging:branch: wip-guits-testing-2024-02-14-1033 2024-02-14T17:24:46.423 INFO:teuthology.packaging:sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:24:46.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-testing-2024-02-14-1033 2024-02-14T17:24:46.655 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1430.g8727ce93 2024-02-14T17:24:46.657 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-14T17:24:46.666 INFO:teuthology.task.internal:no buildpackages task found 2024-02-14T17:24:46.666 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-14T17:24:46.743 INFO:teuthology.task.internal:Saving configuration 2024-02-14T17:24:46.755 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-14T17:24:46.794 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T17:24:46.818 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 17:19:19.975016', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMsvxKdQsDa8qMY7n8wMF12iX6/2AeZvMJA+NH0/z+tBOwWZPHhgYwtuPScajjHIMpOmxa7cWAqHdYjT3yJ+Me0='} 2024-02-14T17:24:46.839 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 17:19:19.976716', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCWf6xgbepP4XQoSuLV9ftXptjCbEYXLJNvnDl37roFCrxZjL69Fc0q1OiLtzlR9cNyA6aDkgHYTXjOPx4mymw='} 2024-02-14T17:24:46.839 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-14T17:24:46.851 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-14T17:24:46.851 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-14T17:24:46.851 INFO:teuthology.run_tasks:Running task console_log... 2024-02-14T17:24:46.961 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f17bba140d0>, signals=[15]) 2024-02-14T17:24:46.962 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-14T17:24:46.970 INFO:teuthology.task.internal:Opening connections... 2024-02-14T17:24:46.970 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-02-14T17:24:46.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:24:47.069 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-02-14T17:24:47.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:24:47.158 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-14T17:24:47.165 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-02-14T17:24:47.183 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-02-14T17:24:47.183 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-02-14T17:24:47.240 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-02-14T17:24:47.240 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-02-14T17:24:47.240 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-02-14T17:24:47.240 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-02-14T17:24:47.240 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-02-14T17:24:47.240 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-02-14T17:24:47.240 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-14T17:24:47.240 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-02-14T17:24:47.240 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-02-14T17:24:47.240 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-14T17:24:47.241 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-02-14T17:24:47.241 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T17:24:47.241 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-14T17:24:47.241 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T17:24:47.241 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-02-14T17:24:47.264 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-02-14T17:24:47.282 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-02-14T17:24:47.282 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-02-14T17:24:47.338 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-14T17:24:47.339 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T17:24:47.340 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-02-14T17:24:47.355 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-14T17:24:47.367 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-14T17:24:47.406 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-14T17:24:47.406 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-02-14T17:24:47.409 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-02-14T17:24:47.425 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-14T17:24:47.469 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-14T17:24:47.470 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-02-14T17:24:47.472 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-02-14T17:24:47.576 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-14T17:24:47.637 INFO:teuthology.run_tasks:Running task kernel... 2024-02-14T17:24:47.663 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-14T17:24:47.663 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-14T17:24:47.663 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-14T17:24:47.664 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-14T17:24:47.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:24:47.681 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-02-14T17:24:47.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:24:47.682 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-02-14T17:24:47.738 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-361.el9.x86_64 2024-02-14T17:24:47.738 INFO:teuthology.task.kernel:Running kernel on smithi178: 5.14.0-361.el9.x86_64 2024-02-14T17:24:47.738 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum install -y kernel 2024-02-14T17:24:47.740 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-361.el9.x86_64 2024-02-14T17:24:47.740 INFO:teuthology.task.kernel:Running kernel on smithi184: 5.14.0-361.el9.x86_64 2024-02-14T17:24:47.740 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum install -y kernel 2024-02-14T17:24:49.837 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-02-14T17:24:49.935 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-14T17:24:51.463 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-02-14T17:24:51.619 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2024-02-14T17:24:53.942 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.0 MB 00:01 2024-02-14T17:24:55.528 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.0 MB 00:03 2024-02-14T17:24:55.868 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-02-14T17:24:57.521 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-02-14T17:25:01.569 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.1 MB 00:02 2024-02-14T17:25:03.111 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 15 kB 00:00 2024-02-14T17:25:03.330 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.1 MB 00:02 2024-02-14T17:25:04.939 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-02-14T17:25:05.162 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-14T17:25:06.723 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-14T17:25:10.062 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-02-14T17:25:11.606 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-14T17:25:12.294 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-14T17:25:12.294 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-14T17:25:12.378 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:25:12.388 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:25:12.388 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-02-14T17:25:12.388 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:25:12.388 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:25:12.388 INFO:teuthology.orchestra.run.smithi178.stdout: kernel x86_64 5.14.0-420.el9 CentOS-BaseOS 6.0 M 2024-02-14T17:25:12.388 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-02-14T17:25:12.388 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core x86_64 5.14.0-420.el9 CentOS-BaseOS 21 M 2024-02-14T17:25:12.388 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules x86_64 5.14.0-420.el9 CentOS-BaseOS 39 M 2024-02-14T17:25:12.388 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core x86_64 5.14.0-420.el9 CentOS-BaseOS 33 M 2024-02-14T17:25:12.388 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:25:12.388 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:25:12.389 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:25:12.389 INFO:teuthology.orchestra.run.smithi178.stdout:Install 4 Packages 2024-02-14T17:25:12.389 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:25:12.389 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 99 M 2024-02-14T17:25:12.389 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 122 M 2024-02-14T17:25:12.389 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:25:12.772 INFO:teuthology.orchestra.run.smithi178.stdout:(1/4): kernel-5.14.0-420.el9.x86_64.rpm 16 MB/s | 6.0 MB 00:00 2024-02-14T17:25:13.788 INFO:teuthology.orchestra.run.smithi178.stdout:(2/4): kernel-modules-5.14.0-420.el9.x86_64.rpm 28 MB/s | 39 MB 00:01 2024-02-14T17:25:13.913 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-14T17:25:13.913 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-14T17:25:13.988 INFO:teuthology.orchestra.run.smithi178.stdout:(3/4): kernel-core-5.14.0-420.el9.x86_64.rpm 13 MB/s | 21 MB 00:01 2024-02-14T17:25:14.016 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:25:14.027 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:25:14.027 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-14T17:25:14.027 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:25:14.027 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:25:14.027 INFO:teuthology.orchestra.run.smithi184.stdout: kernel x86_64 5.14.0-420.el9 CentOS-BaseOS 6.0 M 2024-02-14T17:25:14.027 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-14T17:25:14.027 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core x86_64 5.14.0-420.el9 CentOS-BaseOS 21 M 2024-02-14T17:25:14.027 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules x86_64 5.14.0-420.el9 CentOS-BaseOS 39 M 2024-02-14T17:25:14.027 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core x86_64 5.14.0-420.el9 CentOS-BaseOS 33 M 2024-02-14T17:25:14.028 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:25:14.028 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:25:14.028 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:25:14.028 INFO:teuthology.orchestra.run.smithi184.stdout:Install 4 Packages 2024-02-14T17:25:14.028 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:25:14.028 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 99 M 2024-02-14T17:25:14.028 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 122 M 2024-02-14T17:25:14.029 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:25:14.408 INFO:teuthology.orchestra.run.smithi184.stdout:(1/4): kernel-5.14.0-420.el9.x86_64.rpm 16 MB/s | 6.0 MB 00:00 2024-02-14T17:25:14.430 INFO:teuthology.orchestra.run.smithi178.stdout:(4/4): kernel-modules-core-5.14.0-420.el9.x86_6 20 MB/s | 33 MB 00:01 2024-02-14T17:25:14.433 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:25:14.433 INFO:teuthology.orchestra.run.smithi178.stdout:Total 49 MB/s | 99 MB 00:02 2024-02-14T17:25:14.434 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:25:14.796 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:25:14.797 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:25:14.950 INFO:teuthology.orchestra.run.smithi184.stdout:(2/4): kernel-core-5.14.0-420.el9.x86_64.rpm 23 MB/s | 21 MB 00:00 2024-02-14T17:25:15.708 INFO:teuthology.orchestra.run.smithi184.stdout:(3/4): kernel-modules-5.14.0-420.el9.x86_64.rpm 23 MB/s | 39 MB 00:01 2024-02-14T17:25:15.825 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:25:15.825 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:25:16.008 INFO:teuthology.orchestra.run.smithi184.stdout:(4/4): kernel-modules-core-5.14.0-420.el9.x86_6 21 MB/s | 33 MB 00:01 2024-02-14T17:25:16.012 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:25:16.012 INFO:teuthology.orchestra.run.smithi184.stdout:Total 50 MB/s | 99 MB 00:01 2024-02-14T17:25:16.012 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:25:16.353 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:25:16.354 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:25:17.374 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:25:17.374 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:25:18.206 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:25:18.888 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-core-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:25:19.030 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:25:19.794 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:25:20.478 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-core-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:25:20.609 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:25:21.820 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:25:22.000 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:25:23.358 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:25:23.538 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:25:26.740 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:25:26.912 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:25:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:25:28.359 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:25:31.673 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:25:33.090 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:26:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:26:00.725 INFO:teuthology.orchestra.run.smithi178.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-14T17:26:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:26:00.796 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:26:01.170 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:26:01.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:26:01.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:26:01.170 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:26:01.599 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:26:01.599 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:26:01.599 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:26:01.599 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-5.14.0-420.el9.x86_64 2024-02-14T17:26:01.599 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core-5.14.0-420.el9.x86_64 2024-02-14T17:26:01.599 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-5.14.0-420.el9.x86_64 2024-02-14T17:26:01.599 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core-5.14.0-420.el9.x86_64 2024-02-14T17:26:01.599 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:26:01.599 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:26:02.165 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:26:02.194 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:26:02.194 INFO:teuthology.orchestra.run.smithi184.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-14T17:26:02.194 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:26:02.265 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:26:02.300 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:26:02.300 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:26:02.300 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-420.el9.x86_64 2024-02-14T17:26:02.300 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-14T17:26:02.300 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-14T17:26:02.301 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-14T17:26:02.301 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi178.front.sepia.ceph.com, path=None, version=distro) 2024-02-14T17:26:02.301 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:26:02.410 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:26:02.410 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:26:02.411 DEBUG:teuthology.orchestra.run.smithi178:> sudo rpm -qi grub2-tools 2024-02-14T17:26:02.445 INFO:teuthology.orchestra.run.smithi178.stdout:Name : grub2-tools 2024-02-14T17:26:02.446 INFO:teuthology.orchestra.run.smithi178.stdout:Epoch : 1 2024-02-14T17:26:02.446 INFO:teuthology.orchestra.run.smithi178.stdout:Version : 2.06 2024-02-14T17:26:02.446 INFO:teuthology.orchestra.run.smithi178.stdout:Release : 68.el9 2024-02-14T17:26:02.446 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: x86_64 2024-02-14T17:26:02.446 INFO:teuthology.orchestra.run.smithi178.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-14T17:26:02.446 INFO:teuthology.orchestra.run.smithi178.stdout:Group : Unspecified 2024-02-14T17:26:02.446 INFO:teuthology.orchestra.run.smithi178.stdout:Size : 8264254 2024-02-14T17:26:02.446 INFO:teuthology.orchestra.run.smithi178.stdout:License : GPLv3+ 2024-02-14T17:26:02.446 INFO:teuthology.orchestra.run.smithi178.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-14T17:26:02.446 INFO:teuthology.orchestra.run.smithi178.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:Packager : builder@centos.org 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:Vendor : CentOS 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:Summary : Support tools for GRUB. 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:Description : 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:hardware devices. 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:26:02.447 INFO:teuthology.orchestra.run.smithi178.stdout:This subpackage provides tools for support of all platforms. 2024-02-14T17:26:02.448 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-14T17:26:02.451 INFO:teuthology.task.kernel:Updating grub on smithi178 to boot 5.14.0-420.el9.x86_64 2024-02-14T17:26:02.452 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-14T17:26:02.695 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:26:02.695 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:26:02.695 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:26:02.695 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:26:03.045 INFO:teuthology.orchestra.run.smithi178.stderr:Generating grub configuration file ... 2024-02-14T17:26:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:26:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:26:03.131 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:26:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-5.14.0-420.el9.x86_64 2024-02-14T17:26:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core-5.14.0-420.el9.x86_64 2024-02-14T17:26:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-5.14.0-420.el9.x86_64 2024-02-14T17:26:03.131 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core-5.14.0-420.el9.x86_64 2024-02-14T17:26:03.132 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:26:03.132 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:26:03.640 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:26:03.765 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:26:03.765 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:26:03.765 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-420.el9.x86_64 2024-02-14T17:26:03.766 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-14T17:26:03.766 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-14T17:26:03.766 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-14T17:26:03.766 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=None, version=distro) 2024-02-14T17:26:03.766 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:26:03.874 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:26:03.874 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:26:03.874 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -qi grub2-tools 2024-02-14T17:26:03.909 INFO:teuthology.orchestra.run.smithi184.stdout:Name : grub2-tools 2024-02-14T17:26:03.909 INFO:teuthology.orchestra.run.smithi184.stdout:Epoch : 1 2024-02-14T17:26:03.909 INFO:teuthology.orchestra.run.smithi184.stdout:Version : 2.06 2024-02-14T17:26:03.909 INFO:teuthology.orchestra.run.smithi184.stdout:Release : 68.el9 2024-02-14T17:26:03.909 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: x86_64 2024-02-14T17:26:03.909 INFO:teuthology.orchestra.run.smithi184.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-14T17:26:03.909 INFO:teuthology.orchestra.run.smithi184.stdout:Group : Unspecified 2024-02-14T17:26:03.909 INFO:teuthology.orchestra.run.smithi184.stdout:Size : 8264254 2024-02-14T17:26:03.909 INFO:teuthology.orchestra.run.smithi184.stdout:License : GPLv3+ 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:Packager : builder@centos.org 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:Vendor : CentOS 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:Summary : Support tools for GRUB. 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:Description : 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout:hardware devices. 2024-02-14T17:26:03.910 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:26:03.911 INFO:teuthology.orchestra.run.smithi184.stdout:This subpackage provides tools for support of all platforms. 2024-02-14T17:26:03.911 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-14T17:26:03.911 INFO:teuthology.task.kernel:Updating grub on smithi184 to boot 5.14.0-420.el9.x86_64 2024-02-14T17:26:03.912 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-14T17:26:04.410 INFO:teuthology.orchestra.run.smithi178.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-14T17:26:04.469 INFO:teuthology.orchestra.run.smithi178.stderr:done 2024-02-14T17:26:04.472 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-02-14T17:26:04.488 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.EQ5QoyS0uu 2024-02-14T17:26:04.488 DEBUG:teuthology.orchestra.run.smithi178:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EQ5QoyS0uu 2024-02-14T17:26:04.558 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.EQ5QoyS0uu 2024-02-14T17:26:04.569 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2024-02-14T17:26:04.700 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.EQ5QoyS0uu is 6KB 2024-02-14T17:26:04.749 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.EQ5QoyS0uu 2024-02-14T17:26:04.763 DEBUG:teuthology.orchestra.run.smithi178:> sudo /bin/ls /boot/loader/entries || true 2024-02-14T17:26:04.828 INFO:teuthology.orchestra.run.smithi178.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-14T17:26:04.829 INFO:teuthology.orchestra.run.smithi178.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-14T17:26:04.829 INFO:teuthology.orchestra.run.smithi178.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-14T17:26:04.829 INFO:teuthology.orchestra.run.smithi178.stdout:15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64.conf 2024-02-14T17:26:04.830 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64 2024-02-14T17:26:04.979 DEBUG:teuthology.orchestra.run.smithi178:> sudo shutdown -r now 2024-02-14T17:26:06.034 INFO:teuthology.orchestra.run.smithi184.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-14T17:26:06.063 INFO:teuthology.orchestra.run.smithi184.stderr:done 2024-02-14T17:26:06.065 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-02-14T17:26:06.088 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.Vd2RCO1GZv 2024-02-14T17:26:06.089 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Vd2RCO1GZv 2024-02-14T17:26:06.165 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.Vd2RCO1GZv 2024-02-14T17:26:06.309 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.Vd2RCO1GZv is 6KB 2024-02-14T17:26:06.358 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.Vd2RCO1GZv 2024-02-14T17:26:06.373 DEBUG:teuthology.orchestra.run.smithi184:> sudo /bin/ls /boot/loader/entries || true 2024-02-14T17:26:06.438 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-14T17:26:06.439 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-14T17:26:06.439 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-14T17:26:06.439 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64.conf 2024-02-14T17:26:06.440 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64 2024-02-14T17:26:06.586 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2024-02-14T17:26:35.008 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-14T17:26:35.009 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-02-14T17:26:35.009 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:26:36.589 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-14T17:26:36.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-02-14T17:26:36.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:26:53.500 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-02-14T17:26:55.040 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2024-02-14T17:27:02.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-02-14T17:27:02.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:27:04.041 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-02-14T17:27:04.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:27:05.564 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-02-14T17:27:07.100 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2024-02-14T17:27:17.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-02-14T17:27:17.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:27:19.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-02-14T17:27:19.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:27:20.640 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-02-14T17:27:35.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-02-14T17:27:35.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:27:50.955 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-02-14T17:27:51.357 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-02-14T17:27:51.358 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-420.el9.x86_64"... 2024-02-14T17:27:51.358 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-02-14T17:27:51.372 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-420.el9.x86_64 2024-02-14T17:27:51.372 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-420.el9.x86_64 vs 5.14.0-420.el9.x86_64 2024-02-14T17:27:51.372 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-14T17:27:51.373 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-14T17:27:52.373 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-14T17:27:52.373 DEBUG:teuthology.orchestra.run.smithi184:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-14T17:27:52.457 INFO:teuthology.orchestra.run.smithi184.stdout:ttyS1 2024-02-14T17:27:52.488 DEBUG:teuthology.parallel:result is None 2024-02-14T17:27:54.172 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-02-14T17:28:12.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-02-14T17:28:12.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:28:12.615 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-02-14T17:28:12.988 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-02-14T17:28:12.988 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-420.el9.x86_64"... 2024-02-14T17:28:12.989 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-02-14T17:28:13.004 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-420.el9.x86_64 2024-02-14T17:28:13.004 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-420.el9.x86_64 vs 5.14.0-420.el9.x86_64 2024-02-14T17:28:13.005 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-14T17:28:13.005 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-14T17:28:14.006 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-14T17:28:14.006 DEBUG:teuthology.orchestra.run.smithi178:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-14T17:28:14.078 INFO:teuthology.orchestra.run.smithi178.stdout:ttyS1 2024-02-14T17:28:14.114 DEBUG:teuthology.parallel:result is None 2024-02-14T17:28:14.115 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-14T17:28:14.122 INFO:teuthology.task.internal:Creating test directory... 2024-02-14T17:28:14.122 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-14T17:28:14.125 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-14T17:28:14.143 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-14T17:28:14.164 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-14T17:28:14.217 INFO:teuthology.task.internal:Creating archive directory... 2024-02-14T17:28:14.217 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-14T17:28:14.220 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-14T17:28:14.272 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-14T17:28:14.278 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-14T17:28:14.278 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-14T17:28:14.280 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-14T17:28:14.314 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:28:14.319 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:28:14.329 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:28:14.333 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:28:14.334 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-14T17:28:14.342 INFO:teuthology.task.internal:Configuring sudo... 2024-02-14T17:28:14.342 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-14T17:28:14.374 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-14T17:28:14.404 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-14T17:28:14.417 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-14T17:28:14.417 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-14T17:28:14.444 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-14T17:28:14.460 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:28:14.541 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:28:14.589 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:28:14.673 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:28:14.715 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:28:14.715 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-14T17:28:14.782 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:28:14.826 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:28:14.880 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:28:14.962 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:28:15.007 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:28:15.007 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-14T17:28:15.074 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-02-14T17:28:15.076 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-02-14T17:28:15.132 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T17:28:15.178 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T17:28:15.615 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-14T17:28:15.626 INFO:teuthology.task.internal:Starting timer... 2024-02-14T17:28:15.626 INFO:teuthology.run_tasks:Running task pcp... 2024-02-14T17:28:15.709 INFO:teuthology.run_tasks:Running task selinux... 2024-02-14T17:28:15.750 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-14T17:28:15.751 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-02-14T17:28:15.811 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-02-14T17:28:15.813 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-02-14T17:28:15.861 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-02-14T17:28:15.863 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-14T17:28:15.863 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-02-14T17:28:15.891 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-02-14T17:28:15.891 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-02-14T17:28:15.908 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-02-14T17:28:15.908 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi178.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-02-14T17:28:15.908 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T17:28:15.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:28:15.961 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T17:28:15.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:28:15.990 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-14T17:28:15.990 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-02-14T17:28:16.035 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-02-14T17:28:16.086 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-14T17:28:16.096 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-14T17:28:16.118 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-14T17:28:16.119 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi178.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-14T17:32:32.521 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi178.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-02-14T17:32:32.523 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-02-14T17:32:32.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:32:32.616 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-02-14T17:32:32.696 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-02-14T17:32:32.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-02-14T17:32:32.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:32:32.792 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-02-14T17:32:32.870 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-02-14T17:32:32.871 INFO:teuthology.run_tasks:Running task clock... 2024-02-14T17:32:32.882 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-14T17:32:32.882 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-14T17:32:32.883 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T17:32:32.886 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-14T17:32:32.886 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T17:32:32.916 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-14T17:32:32.935 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-14T17:32:32.943 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-14T17:32:32.960 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-14T17:32:32.963 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-02-14T17:32:32.978 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-02-14T17:32:32.989 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-02-14T17:32:32.993 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-14T17:32:33.003 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-02-14T17:32:33.010 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-14T17:32:33.019 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-14T17:32:33.035 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-14T17:32:33.066 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-02-14T17:32:33.070 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T17:32:33.070 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-02-14T17:32:33.070 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:32:33.070 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:32:33.070 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:32:33.070 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:32:33.090 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-02-14T17:32:33.093 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T17:32:33.093 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-02-14T17:32:33.093 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:32:33.093 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:32:33.093 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:32:33.093 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:32:33.094 INFO:teuthology.run_tasks:Running task pexec... 2024-02-14T17:32:33.103 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-14T17:32:33.104 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi178.front.sepia.ceph.com 2024-02-14T17:32:33.104 DEBUG:teuthology.orchestra.run.smithi178:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-14T17:32:33.104 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi184.front.sepia.ceph.com 2024-02-14T17:32:33.104 DEBUG:teuthology.orchestra.run.smithi184:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-14T17:32:33.710 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:49 ago on Wed 14 Feb 2024 05:31:44 PM UTC. 2024-02-14T17:32:33.731 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:50 ago on Wed 14 Feb 2024 05:31:43 PM UTC. 2024-02-14T17:32:33.870 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:32:33.872 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:32:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-02-14T17:32:33.872 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:32:33.872 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:32:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-14T17:32:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:32:33.872 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:32:33.872 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:32:33.872 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-02-14T17:32:33.872 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:32:33.873 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 3.1 M 2024-02-14T17:32:33.873 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 9.8 M 2024-02-14T17:32:33.873 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:32:33.893 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:32:33.911 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:32:33.911 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-02-14T17:32:33.911 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:32:33.911 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:32:33.912 INFO:teuthology.orchestra.run.smithi184.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-14T17:32:33.912 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:32:33.912 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:32:33.912 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:32:33.912 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-14T17:32:33.912 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:32:33.912 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 3.1 M 2024-02-14T17:32:33.912 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 9.8 M 2024-02-14T17:32:33.912 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:32:34.088 INFO:teuthology.orchestra.run.smithi178.stdout:runc-1.1.12-1.el9.x86_64.rpm 15 MB/s | 3.1 MB 00:00 2024-02-14T17:32:34.089 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:32:34.089 INFO:teuthology.orchestra.run.smithi178.stdout:Total 14 MB/s | 3.1 MB 00:00 2024-02-14T17:32:34.089 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:32:34.094 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:32:34.095 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:32:34.110 INFO:teuthology.orchestra.run.smithi184.stdout:runc-1.1.12-1.el9.x86_64.rpm 16 MB/s | 3.1 MB 00:00 2024-02-14T17:32:34.110 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:32:34.110 INFO:teuthology.orchestra.run.smithi184.stdout:Total 16 MB/s | 3.1 MB 00:00 2024-02-14T17:32:34.110 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:32:34.116 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:32:34.116 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:32:34.166 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:32:34.167 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:32:34.188 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:32:34.188 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:32:34.369 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:32:34.381 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:32:34.470 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:32:34.475 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:32:35.056 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:32:35.098 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:32:35.393 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:32:35.393 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:32:35.393 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:32:35.394 INFO:teuthology.orchestra.run.smithi184.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-14T17:32:35.394 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:32:35.394 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-14T17:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:32:35.663 DEBUG:teuthology.parallel:result is None 2024-02-14T17:32:35.729 DEBUG:teuthology.parallel:result is None 2024-02-14T17:32:35.730 INFO:teuthology.run_tasks:Running task install... 2024-02-14T17:32:35.740 DEBUG:teuthology.task.install:project ceph 2024-02-14T17:32:35.740 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1'}, 'extra_packages': ['cephadm']} 2024-02-14T17:32:35.740 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1'} 2024-02-14T17:32:35.740 INFO:teuthology.task.install:Using flavor: default 2024-02-14T17:32:35.751 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-14T17:32:35.751 INFO:teuthology.task.install:extra packages: [] 2024-02-14T17:32:35.751 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1', 'tag': None, 'wait_for_package': False} 2024-02-14T17:32:35.753 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:32:35.755 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1', 'tag': None, 'wait_for_package': False} 2024-02-14T17:32:35.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:32:35.977 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/ 2024-02-14T17:32:35.978 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1430.g8727ce93 2024-02-14T17:32:35.984 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/ 2024-02-14T17:32:35.984 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1430.g8727ce93 2024-02-14T17:32:36.089 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-14T17:32:36.089 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:32:36.089 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-14T17:32:36.097 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-14T17:32:36.098 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:32:36.098 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-14T17:32:36.121 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-14T17:32:36.121 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8727ce939fc044a98f215e29696bb1433f5758d1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-14T17:32:36.125 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-14T17:32:36.125 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8727ce939fc044a98f215e29696bb1433f5758d1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-14T17:32:36.189 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-14T17:32:36.192 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-14T17:32:36.271 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-14T17:32:36.310 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-14T17:32:36.335 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-02-14T17:32:36.337 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-02-14T17:32:36.352 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-02-14T17:32:36.354 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-02-14T17:32:36.590 INFO:teuthology.orchestra.run.smithi184.stdout:74 files removed 2024-02-14T17:32:36.590 INFO:teuthology.orchestra.run.smithi178.stdout:74 files removed 2024-02-14T17:32:36.618 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw 2024-02-14T17:32:36.621 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw 2024-02-14T17:32:37.356 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 195 kB/s | 82 kB 00:00 2024-02-14T17:32:37.379 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 186 kB/s | 82 kB 00:00 2024-02-14T17:32:37.746 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-02-14T17:32:37.781 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-02-14T17:32:38.122 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-02-14T17:32:38.149 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-02-14T17:32:38.525 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-02-14T17:32:38.584 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-02-14T17:32:39.908 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS-9 - BaseOS 7.7 MB/s | 2.0 MB 00:00 2024-02-14T17:32:40.101 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2024-02-14T17:32:41.242 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.0 MB 00:00 2024-02-14T17:32:41.924 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-02-14T17:32:45.210 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-02-14T17:32:46.125 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 19 MB 00:03 2024-02-14T17:32:50.736 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.1 MB 00:02 2024-02-14T17:32:50.842 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.1 MB 00:01 2024-02-14T17:32:52.260 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 15 kB 00:00 2024-02-14T17:32:53.353 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 15 kB 00:01 2024-02-14T17:32:54.030 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-14T17:32:55.304 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-14T17:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-02-14T17:33:00.171 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-14T17:33:01.514 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:33:01.516 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:01.516 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-02-14T17:33:01.516 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:01.516 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:33:01.516 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 9.7 M 2024-02-14T17:33:01.516 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-02-14T17:33:01.516 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.4 M 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.1 M 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 5.3 M 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 20 M 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 25 k 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 743 k 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-14T17:33:01.517 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-14T17:33:01.525 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 499 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 5.1 M 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 45 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 131 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 162 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 320 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 299 k 2024-02-14T17:33:01.526 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 99 k 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout:Install 30 Packages 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 2 Packages 2024-02-14T17:33:01.527 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:01.528 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 58 M 2024-02-14T17:33:01.528 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:33:02.800 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:33:02.803 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:33:02.803 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-14T17:33:02.803 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:33:02.803 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:33:02.803 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 9.7 M 2024-02-14T17:33:02.803 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-02-14T17:33:02.803 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.4 M 2024-02-14T17:33:02.803 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.1 M 2024-02-14T17:33:02.803 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-14T17:33:02.803 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options 2024-02-14T17:33:02.803 INFO:teuthology.orchestra.run.smithi184.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 5.3 M 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 20 M 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 25 k 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 743 k 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 499 k 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-14T17:33:02.804 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 5.1 M 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 45 k 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 131 k 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 162 k 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 320 k 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 299 k 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 99 k 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-14T17:33:02.805 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-14T17:33:02.806 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-14T17:33:02.806 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-14T17:33:02.806 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-02-14T17:33:02.806 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-14T17:33:02.806 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:33:02.806 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:33:02.806 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:33:02.806 INFO:teuthology.orchestra.run.smithi184.stdout:Install 30 Packages 2024-02-14T17:33:02.806 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 2 Packages 2024-02-14T17:33:02.806 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:33:02.807 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 58 M 2024-02-14T17:33:02.807 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:33:02.912 INFO:teuthology.orchestra.run.smithi178.stdout:(1/32): ceph-base-19.0.0-1430.g8727ce93.el9.x86 4.3 MB/s | 5.3 MB 00:01 2024-02-14T17:33:03.045 INFO:teuthology.orchestra.run.smithi178.stdout:(2/32): ceph-selinux-19.0.0-1430.g8727ce93.el9. 189 kB/s | 25 kB 00:00 2024-02-14T17:33:03.229 INFO:teuthology.orchestra.run.smithi178.stdout:(3/32): ceph-common-19.0.0-1430.g8727ce93.el9.x 13 MB/s | 20 MB 00:01 2024-02-14T17:33:03.271 INFO:teuthology.orchestra.run.smithi178.stdout:(4/32): libcephfs2-19.0.0-1430.g8727ce93.el9.x8 3.2 MB/s | 743 kB 00:00 2024-02-14T17:33:03.304 INFO:teuthology.orchestra.run.smithi178.stdout:(5/32): libradosstriper1-19.0.0-1430.g8727ce93. 6.5 MB/s | 499 kB 00:00 2024-02-14T17:33:03.422 INFO:teuthology.orchestra.run.smithi178.stdout:(6/32): ceph-radosgw-19.0.0-1430.g8727ce93.el9. 5.6 MB/s | 9.7 MB 00:01 2024-02-14T17:33:03.447 INFO:teuthology.orchestra.run.smithi178.stdout:(7/32): python3-ceph-argparse-19.0.0-1430.g8727 317 kB/s | 45 kB 00:00 2024-02-14T17:33:03.514 INFO:teuthology.orchestra.run.smithi178.stdout:(8/32): python3-ceph-common-19.0.0-1430.g8727ce 1.4 MB/s | 131 kB 00:00 2024-02-14T17:33:03.539 INFO:teuthology.orchestra.run.smithi178.stdout:(9/32): python3-cephfs-19.0.0-1430.g8727ce93.el 1.7 MB/s | 162 kB 00:00 2024-02-14T17:33:03.614 INFO:teuthology.orchestra.run.smithi178.stdout:(10/32): python3-rados-19.0.0-1430.g8727ce93.el 3.1 MB/s | 320 kB 00:00 2024-02-14T17:33:03.640 INFO:teuthology.orchestra.run.smithi178.stdout:(11/32): python3-rbd-19.0.0-1430.g8727ce93.el9. 2.9 MB/s | 299 kB 00:00 2024-02-14T17:33:03.715 INFO:teuthology.orchestra.run.smithi178.stdout:(12/32): python3-rgw-19.0.0-1430.g8727ce93.el9. 991 kB/s | 99 kB 00:00 2024-02-14T17:33:03.757 INFO:teuthology.orchestra.run.smithi178.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 905 kB/s | 106 kB 00:00 2024-02-14T17:33:03.790 INFO:teuthology.orchestra.run.smithi178.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-02-14T17:33:03.816 INFO:teuthology.orchestra.run.smithi178.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 756 kB/s | 44 kB 00:00 2024-02-14T17:33:03.883 INFO:teuthology.orchestra.run.smithi178.stdout:(16/32): librgw2-19.0.0-1430.g8727ce93.el9.x86_ 8.3 MB/s | 5.1 MB 00:00 2024-02-14T17:33:03.916 INFO:teuthology.orchestra.run.smithi178.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 5.1 MB/s | 661 kB 00:00 2024-02-14T17:33:03.942 INFO:teuthology.orchestra.run.smithi178.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 282 kB 00:00 2024-02-14T17:33:03.967 INFO:teuthology.orchestra.run.smithi178.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 497 kB/s | 41 kB 00:00 2024-02-14T17:33:03.992 INFO:teuthology.orchestra.run.smithi178.stdout:(20/32): qatlib-23.11.0-1.el9.x86_64.rpm 2.8 MB/s | 215 kB 00:00 2024-02-14T17:33:04.017 INFO:teuthology.orchestra.run.smithi178.stdout:(21/32): qatlib-service-23.11.0-1.el9.x86_64.rp 676 kB/s | 34 kB 00:00 2024-02-14T17:33:04.043 INFO:teuthology.orchestra.run.smithi178.stdout:(22/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 577 kB/s | 43 kB 00:00 2024-02-14T17:33:04.218 INFO:teuthology.orchestra.run.smithi178.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 149 kB/s | 26 kB 00:00 2024-02-14T17:33:04.296 INFO:teuthology.orchestra.run.smithi184.stdout:(1/32): ceph-base-19.0.0-1430.g8727ce93.el9.x86 4.0 MB/s | 5.3 MB 00:01 2024-02-14T17:33:04.330 INFO:teuthology.orchestra.run.smithi184.stdout:(2/32): ceph-selinux-19.0.0-1430.g8727ce93.el9. 758 kB/s | 25 kB 00:00 2024-02-14T17:33:04.343 INFO:teuthology.orchestra.run.smithi178.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-02-14T17:33:04.369 INFO:teuthology.orchestra.run.smithi178.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 821 kB/s | 309 kB 00:00 2024-02-14T17:33:04.411 INFO:teuthology.orchestra.run.smithi178.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-02-14T17:33:04.497 INFO:teuthology.orchestra.run.smithi184.stdout:(3/32): libcephfs2-19.0.0-1430.g8727ce93.el9.x8 4.4 MB/s | 743 kB 00:00 2024-02-14T17:33:04.527 INFO:teuthology.orchestra.run.smithi178.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-02-14T17:33:04.561 INFO:teuthology.orchestra.run.smithi178.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-02-14T17:33:04.614 INFO:teuthology.orchestra.run.smithi184.stdout:(4/32): libradosstriper1-19.0.0-1430.g8727ce93. 4.2 MB/s | 499 kB 00:00 2024-02-14T17:33:04.628 INFO:teuthology.orchestra.run.smithi178.stdout:(29/32): libarrow-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 4.4 MB 00:00 2024-02-14T17:33:04.712 INFO:teuthology.orchestra.run.smithi178.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-02-14T17:33:05.198 INFO:teuthology.orchestra.run.smithi184.stdout:(5/32): ceph-common-19.0.0-1430.g8727ce93.el9.x 8.9 MB/s | 20 MB 00:02 2024-02-14T17:33:05.237 INFO:teuthology.orchestra.run.smithi178.stdout:(31/32): librados2-19.0.0-1430.g8727ce93.el9.x8 5.0 MB/s | 3.4 MB 00:00 2024-02-14T17:33:05.248 INFO:teuthology.orchestra.run.smithi184.stdout:(6/32): python3-ceph-argparse-19.0.0-1430.g8727 908 kB/s | 45 kB 00:00 2024-02-14T17:33:05.287 INFO:teuthology.orchestra.run.smithi178.stdout:(32/32): librbd1-19.0.0-1430.g8727ce93.el9.x86_ 4.8 MB/s | 3.1 MB 00:00 2024-02-14T17:33:05.290 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:33:05.290 INFO:teuthology.orchestra.run.smithi178.stdout:Total 15 MB/s | 58 MB 00:03 2024-02-14T17:33:05.290 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:33:05.306 INFO:teuthology.orchestra.run.smithi184.stdout:(7/32): python3-ceph-common-19.0.0-1430.g8727ce 2.2 MB/s | 131 kB 00:00 2024-02-14T17:33:05.325 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:33:05.325 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:33:05.365 INFO:teuthology.orchestra.run.smithi184.stdout:(8/32): python3-cephfs-19.0.0-1430.g8727ce93.el 2.7 MB/s | 162 kB 00:00 2024-02-14T17:33:05.457 INFO:teuthology.orchestra.run.smithi184.stdout:(9/32): librgw2-19.0.0-1430.g8727ce93.el9.x86_6 6.0 MB/s | 5.1 MB 00:00 2024-02-14T17:33:05.491 INFO:teuthology.orchestra.run.smithi184.stdout:(10/32): python3-rados-19.0.0-1430.g8727ce93.el 2.5 MB/s | 320 kB 00:00 2024-02-14T17:33:05.516 INFO:teuthology.orchestra.run.smithi184.stdout:(11/32): python3-rbd-19.0.0-1430.g8727ce93.el9. 5.0 MB/s | 299 kB 00:00 2024-02-14T17:33:05.541 INFO:teuthology.orchestra.run.smithi184.stdout:(12/32): python3-rgw-19.0.0-1430.g8727ce93.el9. 1.9 MB/s | 99 kB 00:00 2024-02-14T17:33:05.658 INFO:teuthology.orchestra.run.smithi184.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 746 kB/s | 106 kB 00:00 2024-02-14T17:33:05.684 INFO:teuthology.orchestra.run.smithi184.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 159 kB 00:00 2024-02-14T17:33:05.709 INFO:teuthology.orchestra.run.smithi184.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 880 kB/s | 44 kB 00:00 2024-02-14T17:33:05.760 INFO:teuthology.orchestra.run.smithi184.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-02-14T17:33:05.784 INFO:teuthology.orchestra.run.smithi184.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-02-14T17:33:05.810 INFO:teuthology.orchestra.run.smithi184.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 828 kB/s | 41 kB 00:00 2024-02-14T17:33:05.812 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:33:05.812 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:33:05.835 INFO:teuthology.orchestra.run.smithi184.stdout:(19/32): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 215 kB 00:00 2024-02-14T17:33:05.886 INFO:teuthology.orchestra.run.smithi184.stdout:(20/32): qatlib-service-23.11.0-1.el9.x86_64.rp 449 kB/s | 34 kB 00:00 2024-02-14T17:33:05.911 INFO:teuthology.orchestra.run.smithi184.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 575 kB/s | 43 kB 00:00 2024-02-14T17:33:06.019 INFO:teuthology.orchestra.run.smithi184.stdout:(22/32): ceph-radosgw-19.0.0-1430.g8727ce93.el9 3.2 MB/s | 9.7 MB 00:03 2024-02-14T17:33:06.314 INFO:teuthology.orchestra.run.smithi184.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 88 kB/s | 26 kB 00:00 2024-02-14T17:33:06.470 INFO:teuthology.orchestra.run.smithi184.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 529 kB/s | 309 kB 00:00 2024-02-14T17:33:06.503 INFO:teuthology.orchestra.run.smithi184.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 257 kB/s | 49 kB 00:00 2024-02-14T17:33:06.564 INFO:teuthology.orchestra.run.smithi184.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-02-14T17:33:06.721 INFO:teuthology.orchestra.run.smithi184.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-02-14T17:33:06.863 INFO:teuthology.orchestra.run.smithi184.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2024-02-14T17:33:07.038 INFO:teuthology.orchestra.run.smithi184.stdout:(29/32): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-02-14T17:33:07.105 INFO:teuthology.orchestra.run.smithi184.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-02-14T17:33:07.240 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:33:07.304 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-02-14T17:33:07.339 INFO:teuthology.orchestra.run.smithi184.stdout:(31/32): librbd1-19.0.0-1430.g8727ce93.el9.x86_ 10 MB/s | 3.1 MB 00:00 2024-02-14T17:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-02-14T17:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-02-14T17:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-14T17:33:07.405 INFO:teuthology.orchestra.run.smithi184.stdout:(32/32): librados2-19.0.0-1430.g8727ce93.el9.x8 6.2 MB/s | 3.4 MB 00:00 2024-02-14T17:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-14T17:33:07.409 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:33:07.410 INFO:teuthology.orchestra.run.smithi184.stdout:Total 13 MB/s | 58 MB 00:04 2024-02-14T17:33:07.410 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:33:07.448 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:33:07.448 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:33:07.601 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-14T17:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-14T17:33:07.934 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:33:07.935 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:33:07.980 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-02-14T17:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:33:08.128 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:33:08.218 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 8/34 2024-02-14T17:33:08.254 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 9/34 2024-02-14T17:33:08.334 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 9/34 2024-02-14T17:33:08.371 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-02-14T17:33:08.416 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9. 11/34 2024-02-14T17:33:08.485 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 12/34 2024-02-14T17:33:08.507 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 13/34 2024-02-14T17:33:08.569 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 13/34 2024-02-14T17:33:08.598 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-02-14T17:33:08.708 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-02-14T17:33:08.744 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-02-14T17:33:08.788 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-02-14T17:33:08.813 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-02-14T17:33:08.834 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-02-14T17:33:08.860 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-02-14T17:33:09.172 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-02-14T17:33:09.222 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 22/34 2024-02-14T17:33:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 22/34 2024-02-14T17:33:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 23/34 2024-02-14T17:33:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-02-14T17:33:09.393 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:33:09.489 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-02-14T17:33:09.519 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-02-14T17:33:09.544 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-02-14T17:33:09.567 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-14T17:33:09.583 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-14T17:33:09.781 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-14T17:33:09.824 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-02-14T17:33:09.840 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-14T17:33:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 26/34 2024-02-14T17:33:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 26/34 2024-02-14T17:33:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 27/34 2024-02-14T17:33:10.027 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x8 28/34 2024-02-14T17:33:10.209 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-02-14T17:33:10.275 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:33:10.370 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:33:10.457 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 8/34 2024-02-14T17:33:10.480 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 9/34 2024-02-14T17:33:10.536 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 9/34 2024-02-14T17:33:10.563 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-02-14T17:33:10.599 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9. 11/34 2024-02-14T17:33:10.667 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 12/34 2024-02-14T17:33:10.691 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 13/34 2024-02-14T17:33:10.737 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 13/34 2024-02-14T17:33:10.764 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-02-14T17:33:10.870 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-02-14T17:33:10.904 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-02-14T17:33:10.940 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-02-14T17:33:10.980 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-02-14T17:33:11.009 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-02-14T17:33:11.042 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-02-14T17:33:11.349 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-02-14T17:33:11.380 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 22/34 2024-02-14T17:33:11.462 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 22/34 2024-02-14T17:33:11.494 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 23/34 2024-02-14T17:33:11.532 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-02-14T17:33:11.984 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-02-14T17:33:12.008 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 26/34 2024-02-14T17:33:12.079 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 26/34 2024-02-14T17:33:12.164 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 27/34 2024-02-14T17:33:12.166 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:33:12.194 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x8 28/34 2024-02-14T17:33:12.207 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:33:12.813 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:33:12.870 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 30/34 2024-02-14T17:33:12.961 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 30/34 2024-02-14T17:33:12.961 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-14T17:33:12.961 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-14T17:33:12.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:13.005 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:33:14.338 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:33:14.374 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:33:14.939 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:33:14.977 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 30/34 2024-02-14T17:33:15.064 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 30/34 2024-02-14T17:33:15.064 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-14T17:33:15.065 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-14T17:33:15.065 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:33:15.145 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:33:22.556 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:33:22.556 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-02-14T17:33:22.556 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-02-14T17:33:22.556 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-02-14T17:33:22.556 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-02-14T17:33:22.556 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-02-14T17:33:22.556 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-02-14T17:33:22.557 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-02-14T17:33:22.557 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:23.469 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 32/34 2024-02-14T17:33:23.493 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 32/34 2024-02-14T17:33:23.493 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:33:23.494 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-14T17:33:23.494 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T17:33:23.494 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T17:33:23.494 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:23.495 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-14T17:33:23.580 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-14T17:33:23.580 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:33:24.753 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:33:24.754 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 1/34 2024-02-14T17:33:24.754 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2/34 2024-02-14T17:33:24.754 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 3/34 2024-02-14T17:33:24.754 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 4/34 2024-02-14T17:33:24.754 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 5/34 2024-02-14T17:33:24.754 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 6/34 2024-02-14T17:33:24.754 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:33:24.754 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9. 8/34 2024-02-14T17:33:24.754 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x8 9/34 2024-02-14T17:33:24.754 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 10/34 2024-02-14T17:33:24.754 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 11/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 12/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 13/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-02-14T17:33:24.755 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-02-14T17:33:24.756 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-02-14T17:33:24.756 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-02-14T17:33:24.756 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-02-14T17:33:24.756 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-02-14T17:33:24.757 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-02-14T17:33:24.757 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-02-14T17:33:24.757 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-02-14T17:33:24.757 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:33:24.757 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-02-14T17:33:24.757 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 33/34 2024-02-14T17:33:24.821 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:33:24.821 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-02-14T17:33:24.821 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-02-14T17:33:24.821 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-02-14T17:33:24.821 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-02-14T17:33:24.821 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-02-14T17:33:24.821 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-02-14T17:33:24.821 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-02-14T17:33:24.821 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:33:25.743 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 32/34 2024-02-14T17:33:25.768 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 32/34 2024-02-14T17:33:25.768 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:33:25.768 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-14T17:33:25.768 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T17:33:25.769 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T17:33:25.769 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:33:25.770 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.819 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-14T17:33:25.820 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-14T17:33:25.820 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-14T17:33:25.820 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.820 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-14T17:33:25.820 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-14T17:33:25.820 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-14T17:33:25.823 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.823 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-14T17:33:25.823 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.823 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-14T17:33:25.823 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-14T17:33:25.823 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-14T17:33:25.823 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: re2-1:20211101-3.el9.x86_64 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:25.824 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:33:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-14T17:33:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:33:26.143 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-test 2024-02-14T17:33:26.742 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:28 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:33:26.914 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:33:26.915 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:26.915 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-02-14T17:33:26.915 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:26.915 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:33:26.915 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 47 M 2024-02-14T17:33:26.915 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-02-14T17:33:26.915 INFO:teuthology.orchestra.run.smithi178.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-14T17:33:26.915 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 163 k 2024-02-14T17:33:26.916 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-14T17:33:26.916 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-14T17:33:26.916 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-14T17:33:26.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:26.916 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:33:26.916 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:26.916 INFO:teuthology.orchestra.run.smithi178.stdout:Install 6 Packages 2024-02-14T17:33:26.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:26.917 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 48 M 2024-02-14T17:33:26.917 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 204 M 2024-02-14T17:33:26.917 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:33:27.007 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:33:27.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 1/34 2024-02-14T17:33:27.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2/34 2024-02-14T17:33:27.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 3/34 2024-02-14T17:33:27.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 4/34 2024-02-14T17:33:27.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 5/34 2024-02-14T17:33:27.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 6/34 2024-02-14T17:33:27.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9. 8/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x8 9/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 10/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 11/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 12/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 13/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-02-14T17:33:27.008 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-02-14T17:33:27.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-02-14T17:33:27.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-02-14T17:33:27.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-02-14T17:33:27.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-02-14T17:33:27.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-02-14T17:33:27.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-02-14T17:33:27.009 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-02-14T17:33:27.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-02-14T17:33:27.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-02-14T17:33:27.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:33:27.010 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-02-14T17:33:27.011 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 33/34 2024-02-14T17:33:27.086 INFO:teuthology.orchestra.run.smithi178.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 300 kB 00:00 2024-02-14T17:33:27.128 INFO:teuthology.orchestra.run.smithi178.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-14T17:33:27.170 INFO:teuthology.orchestra.run.smithi178.stdout:(3/6): libcephsqlite-19.0.0-1430.g8727ce93.el9. 647 kB/s | 163 kB 00:00 2024-02-14T17:33:27.195 INFO:teuthology.orchestra.run.smithi178.stdout:(4/6): jq-1.6-16.el9.x86_64.rpm 2.7 MB/s | 186 kB 00:00 2024-02-14T17:33:27.229 INFO:teuthology.orchestra.run.smithi178.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-02-14T17:33:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:33:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:33:28.187 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-02-14T17:33:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.187 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:33:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-14T17:33:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.187 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-14T17:33:28.188 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-3.el9.x86_64 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:33:28.189 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:33:28.456 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-test 2024-02-14T17:33:29.033 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:29 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:33:29.210 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 47 M 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 163 k 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:33:29.211 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:33:29.212 INFO:teuthology.orchestra.run.smithi184.stdout:Install 6 Packages 2024-02-14T17:33:29.212 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:33:29.212 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 48 M 2024-02-14T17:33:29.212 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 204 M 2024-02-14T17:33:29.212 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:33:29.412 INFO:teuthology.orchestra.run.smithi178.stdout:(6/6): ceph-test-19.0.0-1430.g8727ce93.el9.x86_ 19 MB/s | 47 MB 00:02 2024-02-14T17:33:29.415 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:33:29.415 INFO:teuthology.orchestra.run.smithi178.stdout:Total 19 MB/s | 48 MB 00:02 2024-02-14T17:33:29.415 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:33:29.436 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:33:29.436 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:33:29.828 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:33:29.829 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:33:30.140 INFO:teuthology.orchestra.run.smithi184.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 323 kB/s | 300 kB 00:00 2024-02-14T17:33:30.399 INFO:teuthology.orchestra.run.smithi184.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 243 kB/s | 63 kB 00:00 2024-02-14T17:33:30.404 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:33:30.486 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-14T17:33:30.519 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-14T17:33:30.563 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-14T17:33:30.616 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-14T17:33:30.640 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 5/6 2024-02-14T17:33:30.641 INFO:teuthology.orchestra.run.smithi184.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 770 kB/s | 186 kB 00:00 2024-02-14T17:33:30.908 INFO:teuthology.orchestra.run.smithi184.stdout:(4/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 813 kB/s | 217 kB 00:00 2024-02-14T17:33:30.933 INFO:teuthology.orchestra.run.smithi184.stdout:(5/6): libcephsqlite-19.0.0-1430.g8727ce93.el9. 95 kB/s | 163 kB 00:01 2024-02-14T17:33:35.065 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 5/6 2024-02-14T17:33:35.146 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 6/6 2024-02-14T17:33:37.631 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 6/6 2024-02-14T17:33:37.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 1/6 2024-02-14T17:33:37.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 2/6 2024-02-14T17:33:37.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-14T17:33:37.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-14T17:33:37.631 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-14T17:33:38.114 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-14T17:33:38.114 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:38.115 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:33:38.115 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:38.115 INFO:teuthology.orchestra.run.smithi178.stdout: jq-1.6-16.el9.x86_64 2024-02-14T17:33:38.115 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:38.115 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-14T17:33:38.115 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-14T17:33:38.115 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-14T17:33:38.115 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:38.115 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:33:38.365 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph 2024-02-14T17:33:38.971 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:40 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:33:39.158 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:33:39.160 INFO:teuthology.orchestra.run.smithi178.stdout:========================================================================================= 2024-02-14T17:33:39.160 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-02-14T17:33:39.160 INFO:teuthology.orchestra.run.smithi178.stdout:========================================================================================= 2024-02-14T17:33:39.160 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:33:39.160 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 6.6 k 2024-02-14T17:33:39.160 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-02-14T17:33:39.160 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 2.1 M 2024-02-14T17:33:39.160 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 1.5 M 2024-02-14T17:33:39.160 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 247 k 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 4.7 M 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 17 M 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-14T17:33:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-14T17:33:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-14T17:33:39.163 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-14T17:33:39.163 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-14T17:33:39.163 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-14T17:33:39.163 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-14T17:33:39.163 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:39.163 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:33:39.163 INFO:teuthology.orchestra.run.smithi178.stdout:========================================================================================= 2024-02-14T17:33:39.163 INFO:teuthology.orchestra.run.smithi178.stdout:Install 36 Packages 2024-02-14T17:33:39.163 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:39.164 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 30 M 2024-02-14T17:33:39.164 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 107 M 2024-02-14T17:33:39.164 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:33:40.014 INFO:teuthology.orchestra.run.smithi178.stdout:(1/36): ceph-19.0.0-1430.g8727ce93.el9.x86_64.r 20 kB/s | 6.6 kB 00:00 2024-02-14T17:33:40.515 INFO:teuthology.orchestra.run.smithi178.stdout:(2/36): ceph-mgr-19.0.0-1430.g8727ce93.el9.x86_ 1.8 MB/s | 1.5 MB 00:00 2024-02-14T17:33:40.757 INFO:teuthology.orchestra.run.smithi178.stdout:(3/36): ceph-mds-19.0.0-1430.g8727ce93.el9.x86_ 2.0 MB/s | 2.1 MB 00:01 2024-02-14T17:33:40.882 INFO:teuthology.orchestra.run.smithi178.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1430.g8727 1.9 MB/s | 247 kB 00:00 2024-02-14T17:33:41.157 INFO:teuthology.orchestra.run.smithi178.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 679 kB/s | 187 kB 00:00 2024-02-14T17:33:41.249 INFO:teuthology.orchestra.run.smithi178.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 144 kB 00:00 2024-02-14T17:33:41.316 INFO:teuthology.orchestra.run.smithi178.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 475 kB/s | 32 kB 00:00 2024-02-14T17:33:41.391 INFO:teuthology.orchestra.run.smithi178.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 629 kB/s | 47 kB 00:00 2024-02-14T17:33:41.516 INFO:teuthology.orchestra.run.smithi178.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 309 kB/s | 39 kB 00:00 2024-02-14T17:33:41.641 INFO:teuthology.orchestra.run.smithi178.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 241 kB 00:00 2024-02-14T17:33:41.700 INFO:teuthology.orchestra.run.smithi178.stdout:(11/36): ceph-mon-19.0.0-1430.g8727ce93.el9.x86 2.8 MB/s | 4.7 MB 00:01 2024-02-14T17:33:41.884 INFO:teuthology.orchestra.run.smithi178.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 4.7 MB/s | 1.1 MB 00:00 2024-02-14T17:33:41.909 INFO:teuthology.orchestra.run.smithi178.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 494 kB/s | 103 kB 00:00 2024-02-14T17:33:41.942 INFO:teuthology.orchestra.run.smithi178.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.1 MB/s | 124 kB 00:00 2024-02-14T17:33:41.968 INFO:teuthology.orchestra.run.smithi178.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 1.9 MB/s | 113 kB 00:00 2024-02-14T17:33:42.001 INFO:teuthology.orchestra.run.smithi178.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 187 kB 00:00 2024-02-14T17:33:42.060 INFO:teuthology.orchestra.run.smithi178.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-02-14T17:33:42.252 INFO:teuthology.orchestra.run.smithi178.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 227 kB/s | 43 kB 00:00 2024-02-14T17:33:42.411 INFO:teuthology.orchestra.run.smithi178.stdout:(19/36): ceph-osd-19.0.0-1430.g8727ce93.el9.x86 8.8 MB/s | 17 MB 00:01 2024-02-14T17:33:42.436 INFO:teuthology.orchestra.run.smithi178.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 348 kB/s | 151 kB 00:00 2024-02-14T17:33:42.461 INFO:teuthology.orchestra.run.smithi178.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 822 kB/s | 172 kB 00:00 2024-02-14T17:33:42.504 INFO:teuthology.orchestra.run.smithi178.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2024-02-14T17:33:42.528 INFO:teuthology.orchestra.run.smithi178.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 264 kB/s | 18 kB 00:00 2024-02-14T17:33:42.587 INFO:teuthology.orchestra.run.smithi178.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 277 kB/s | 23 kB 00:00 2024-02-14T17:33:42.613 INFO:teuthology.orchestra.run.smithi178.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 231 kB/s | 19 kB 00:00 2024-02-14T17:33:42.680 INFO:teuthology.orchestra.run.smithi178.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 213 kB/s | 20 kB 00:00 2024-02-14T17:33:42.705 INFO:teuthology.orchestra.run.smithi178.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 504 kB/s | 46 kB 00:00 2024-02-14T17:33:42.763 INFO:teuthology.orchestra.run.smithi178.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 944 kB/s | 79 kB 00:00 2024-02-14T17:33:42.831 INFO:teuthology.orchestra.run.smithi178.stdout:(29/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-02-14T17:33:42.856 INFO:teuthology.orchestra.run.smithi178.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 179 kB/s | 16 kB 00:00 2024-02-14T17:33:42.881 INFO:teuthology.orchestra.run.smithi178.stdout:(31/36): python3-cherrypy-18.6.1-2.el9.noarch.r 764 kB/s | 358 kB 00:00 2024-02-14T17:33:42.906 INFO:teuthology.orchestra.run.smithi178.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-02-14T17:33:42.923 INFO:teuthology.orchestra.run.smithi178.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 533 kB/s | 36 kB 00:00 2024-02-14T17:33:42.982 INFO:teuthology.orchestra.run.smithi178.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-02-14T17:33:42.999 INFO:teuthology.orchestra.run.smithi178.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 266 kB/s | 20 kB 00:00 2024-02-14T17:33:43.024 INFO:teuthology.orchestra.run.smithi178.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-02-14T17:33:43.027 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:33:43.027 INFO:teuthology.orchestra.run.smithi178.stdout:Total 7.7 MB/s | 30 MB 00:03 2024-02-14T17:33:43.045 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:33:43.088 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:33:43.088 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:33:43.517 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:33:43.518 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:33:44.883 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:33:44.959 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-14T17:33:44.992 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-14T17:33:45.029 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-14T17:33:45.099 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-14T17:33:45.152 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-14T17:33:45.227 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-14T17:33:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-14T17:33:45.404 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-14T17:33:45.455 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-14T17:33:45.527 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-14T17:33:45.588 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-14T17:33:45.635 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-14T17:33:45.683 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-14T17:33:45.740 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-14T17:33:45.923 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-14T17:33:45.973 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-14T17:33:46.018 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-14T17:33:46.054 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-14T17:33:46.084 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-14T17:33:46.160 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-14T17:33:46.195 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-14T17:33:46.228 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-14T17:33:46.269 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-14T17:33:46.303 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-14T17:33:46.339 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-14T17:33:46.427 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-14T17:33:46.460 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-14T17:33:46.533 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-14T17:33:46.670 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-14T17:33:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-14T17:33:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9. 31/36 2024-02-14T17:33:46.990 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 32/36 2024-02-14T17:33:47.016 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 32/36 2024-02-14T17:33:47.016 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:33:47.016 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-14T17:33:47.016 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T17:33:47.016 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T17:33:47.016 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:48.449 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 33/36 2024-02-14T17:33:48.476 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 33/36 2024-02-14T17:33:48.476 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:33:48.476 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-14T17:33:48.476 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T17:33:48.476 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T17:33:48.477 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 34/36 2024-02-14T17:33:48.940 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 34/36 2024-02-14T17:33:48.940 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:33:48.940 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-14T17:33:48.940 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T17:33:48.940 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T17:33:48.940 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:49.151 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 35/36 2024-02-14T17:33:49.174 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 35/36 2024-02-14T17:33:49.174 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:33:49.174 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-14T17:33:49.174 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T17:33:49.174 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T17:33:49.175 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:49.240 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 36/36 2024-02-14T17:33:51.117 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 36/36 2024-02-14T17:33:51.118 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 1/36 2024-02-14T17:33:51.118 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 2/36 2024-02-14T17:33:51.118 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 3/36 2024-02-14T17:33:51.118 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 4/36 2024-02-14T17:33:51.118 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 5/36 2024-02-14T17:33:51.118 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9. 6/36 2024-02-14T17:33:51.118 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-14T17:33:51.118 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-14T17:33:51.118 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-14T17:33:51.118 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-14T17:33:51.119 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-14T17:33:51.119 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-14T17:33:51.119 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-14T17:33:51.119 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-14T17:33:51.119 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-14T17:33:51.119 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-14T17:33:51.120 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-14T17:33:51.120 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-14T17:33:51.120 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-14T17:33:51.120 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-14T17:33:51.120 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-14T17:33:51.120 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-14T17:33:51.120 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-14T17:33:51.120 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-14T17:33:51.120 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-14T17:33:51.122 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-14T17:33:51.122 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-14T17:33:51.122 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-14T17:33:51.122 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-14T17:33:51.122 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-14T17:33:51.122 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-14T17:33:51.122 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-14T17:33:51.122 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-14T17:33:51.122 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-14T17:33:51.122 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-14T17:33:52.056 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-14T17:33:52.056 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:52.056 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:33:52.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:52.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:52.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:52.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:33:52.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:52.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: lua-5.4.4-4.el9.x86_64 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-14T17:33:52.057 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-14T17:33:52.058 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-14T17:33:52.059 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-14T17:33:52.059 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:52.059 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:33:52.403 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-base 2024-02-14T17:33:52.981 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:54 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:33:53.093 INFO:teuthology.orchestra.run.smithi178.stdout:Package ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:33:53.143 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:33:53.144 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-02-14T17:33:53.145 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:33:53.221 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install cephadm 2024-02-14T17:33:53.798 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:55 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:33:53.966 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:33:53.966 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:53.967 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-02-14T17:33:53.967 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:53.967 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:33:53.967 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 748 k 2024-02-14T17:33:53.967 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:53.967 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:33:53.967 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:53.968 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-02-14T17:33:53.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:53.968 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 748 k 2024-02-14T17:33:53.968 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 754 k 2024-02-14T17:33:53.968 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:33:54.290 INFO:teuthology.orchestra.run.smithi178.stdout:cephadm-19.0.0-1430.g8727ce93.el9.noarch.rpm 2.3 MB/s | 748 kB 00:00 2024-02-14T17:33:54.291 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:33:54.291 INFO:teuthology.orchestra.run.smithi178.stdout:Total 2.3 MB/s | 748 kB 00:00 2024-02-14T17:33:54.291 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:33:54.296 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:33:54.296 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:33:54.301 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:33:54.302 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:33:54.379 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:33:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:33:54.920 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:33:55.343 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:33:55.672 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:33:55.672 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:55.672 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:33:55.672 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:33:55.672 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:55.672 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:33:55.890 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-immutable-object-cache 2024-02-14T17:33:56.130 INFO:teuthology.orchestra.run.smithi184.stdout:(6/6): ceph-test-19.0.0-1430.g8727ce93.el9.x86_ 1.8 MB/s | 47 MB 00:26 2024-02-14T17:33:56.134 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:33:56.134 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.8 MB/s | 48 MB 00:26 2024-02-14T17:33:56.134 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:33:56.154 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:33:56.154 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:33:56.467 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:58 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:33:56.552 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:33:56.552 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:33:56.636 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:33:56.636 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:56.636 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-02-14T17:33:56.636 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:56.637 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:33:56.637 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 145 k 2024-02-14T17:33:56.637 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:56.637 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:33:56.637 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:33:56.637 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-02-14T17:33:56.637 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:56.637 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 145 k 2024-02-14T17:33:56.637 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 436 k 2024-02-14T17:33:56.637 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:33:57.169 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:33:57.231 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-14T17:33:57.262 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-14T17:33:57.312 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-14T17:33:57.341 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-immutable-object-cache-19.0.0-1430.g8727ce 207 kB/s | 145 kB 00:00 2024-02-14T17:33:57.341 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:33:57.341 INFO:teuthology.orchestra.run.smithi178.stdout:Total 207 kB/s | 145 kB 00:00 2024-02-14T17:33:57.341 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:33:57.347 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:33:57.347 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:33:57.358 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-14T17:33:57.383 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 5/6 2024-02-14T17:33:57.397 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:33:57.398 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:33:57.557 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:33:57.604 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:33:57.628 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:33:57.629 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:33:57.629 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-14T17:33:57.629 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:58.711 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:33:58.711 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:58.711 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:33:58.711 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:33:58.711 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:33:58.711 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:33:58.948 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr 2024-02-14T17:33:59.539 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:01 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:33:59.651 INFO:teuthology.orchestra.run.smithi178.stdout:Package ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:33:59.703 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:33:59.704 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-02-14T17:33:59.704 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:33:59.783 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-dashboard 2024-02-14T17:34:00.360 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:02 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:00.528 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:00.529 INFO:teuthology.orchestra.run.smithi178.stdout:====================================================================================== 2024-02-14T17:34:00.529 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-02-14T17:34:00.529 INFO:teuthology.orchestra.run.smithi178.stdout:====================================================================================== 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 1.8 M 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 31 k 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 15 k 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-14T17:34:00.530 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:00.531 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:34:00.531 INFO:teuthology.orchestra.run.smithi178.stdout:====================================================================================== 2024-02-14T17:34:00.531 INFO:teuthology.orchestra.run.smithi178.stdout:Install 12 Packages 2024-02-14T17:34:00.531 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:00.531 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 6.9 M 2024-02-14T17:34:00.531 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 26 M 2024-02-14T17:34:00.531 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:34:00.982 INFO:teuthology.orchestra.run.smithi178.stdout:(1/12): ceph-prometheus-alerts-19.0.0-1430.g872 81 kB/s | 15 kB 00:00 2024-02-14T17:34:01.174 INFO:teuthology.orchestra.run.smithi178.stdout:(2/12): ceph-grafana-dashboards-19.0.0-1430.g87 83 kB/s | 31 kB 00:00 2024-02-14T17:34:01.299 INFO:teuthology.orchestra.run.smithi178.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.2 MB/s | 1.0 MB 00:00 2024-02-14T17:34:01.316 INFO:teuthology.orchestra.run.smithi178.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.6 MB/s | 238 kB 00:00 2024-02-14T17:34:01.775 INFO:teuthology.orchestra.run.smithi178.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 1.8 MB/s | 863 kB 00:00 2024-02-14T17:34:01.798 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 5/6 2024-02-14T17:34:01.809 INFO:teuthology.orchestra.run.smithi178.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.1 MB/s | 548 kB 00:00 2024-02-14T17:34:01.875 INFO:teuthology.orchestra.run.smithi178.stdout:(7/12): ceph-mgr-dashboard-19.0.0-1430.g8727ce9 1.6 MB/s | 1.8 MB 00:01 2024-02-14T17:34:01.920 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 6/6 2024-02-14T17:34:02.009 INFO:teuthology.orchestra.run.smithi178.stdout:(8/12): grpc-data-1.46.7-10.el9.noarch.rpm 83 kB/s | 19 kB 00:00 2024-02-14T17:34:02.068 INFO:teuthology.orchestra.run.smithi178.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 526 kB/s | 31 kB 00:00 2024-02-14T17:34:02.176 INFO:teuthology.orchestra.run.smithi178.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 483 kB/s | 144 kB 00:00 2024-02-14T17:34:02.201 INFO:teuthology.orchestra.run.smithi178.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.4 MB/s | 188 kB 00:00 2024-02-14T17:34:03.276 INFO:teuthology.orchestra.run.smithi178.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 1.4 MB/s | 2.0 MB 00:01 2024-02-14T17:34:03.282 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:03.282 INFO:teuthology.orchestra.run.smithi178.stdout:Total 2.5 MB/s | 6.9 MB 00:02 2024-02-14T17:34:03.300 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:34:03.317 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:34:03.317 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:34:03.487 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:34:03.487 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:34:04.003 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:34:04.086 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-14T17:34:04.121 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-14T17:34:04.167 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-14T17:34:04.197 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-14T17:34:04.294 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-14T17:34:04.352 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 6/6 2024-02-14T17:34:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 1/6 2024-02-14T17:34:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 2/6 2024-02-14T17:34:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-14T17:34:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-14T17:34:04.444 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-14T17:34:04.445 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-14T17:34:04.516 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-14T17:34:04.560 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-14T17:34:04.589 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-14T17:34:04.620 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9 10/12 2024-02-14T17:34:04.787 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-14T17:34:04.787 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:04.787 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:34:04.787 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:04.787 INFO:teuthology.orchestra.run.smithi184.stdout: jq-1.6-16.el9.x86_64 2024-02-14T17:34:04.787 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:04.787 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-14T17:34:04.787 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-14T17:34:04.787 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-14T17:34:04.787 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:04.788 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:34:04.862 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el 11/12 2024-02-14T17:34:04.895 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 12/12 2024-02-14T17:34:05.015 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph 2024-02-14T17:34:05.596 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:05 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:34:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 12/12 2024-02-14T17:34:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el 1/12 2024-02-14T17:34:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 2/12 2024-02-14T17:34:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9 3/12 2024-02-14T17:34:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-14T17:34:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-14T17:34:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-14T17:34:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-14T17:34:05.760 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-14T17:34:05.760 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-14T17:34:05.760 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-14T17:34:05.760 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-14T17:34:05.773 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:34:05.775 INFO:teuthology.orchestra.run.smithi184.stdout:========================================================================================= 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout:========================================================================================= 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 6.6 k 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 2.1 M 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 1.5 M 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 247 k 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 4.7 M 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 17 M 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-14T17:34:05.776 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-14T17:34:05.777 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:05.778 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:34:05.779 INFO:teuthology.orchestra.run.smithi184.stdout:========================================================================================= 2024-02-14T17:34:05.779 INFO:teuthology.orchestra.run.smithi184.stdout:Install 36 Packages 2024-02-14T17:34:05.779 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:05.779 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 30 M 2024-02-14T17:34:05.779 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 107 M 2024-02-14T17:34:05.779 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:34:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-14T17:34:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-14T17:34:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-14T17:34:06.125 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:06.125 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:06.290 INFO:teuthology.orchestra.run.smithi184.stdout:(1/36): ceph-19.0.0-1430.g8727ce93.el9.x86_64.r 24 kB/s | 6.6 kB 00:00 2024-02-14T17:34:06.408 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-14T17:34:06.615 INFO:teuthology.orchestra.run.smithi184.stdout:(2/36): ceph-mds-19.0.0-1430.g8727ce93.el9.x86_ 3.5 MB/s | 2.1 MB 00:00 2024-02-14T17:34:06.988 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:08 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:07.158 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:07.159 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================ 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================ 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 7.4 M 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-14T17:34:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-14T17:34:07.161 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-14T17:34:07.161 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-14T17:34:07.161 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:07.161 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:34:07.161 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================ 2024-02-14T17:34:07.161 INFO:teuthology.orchestra.run.smithi178.stdout:Install 12 Packages 2024-02-14T17:34:07.161 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:07.162 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 37 M 2024-02-14T17:34:07.162 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 209 M 2024-02-14T17:34:07.162 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:34:07.633 INFO:teuthology.orchestra.run.smithi178.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 127 kB/s | 32 kB 00:00 2024-02-14T17:34:07.699 INFO:teuthology.orchestra.run.smithi184.stdout:(3/36): ceph-osd-19.0.0-1430.g8727ce93.el9.x86_ 15 MB/s | 17 MB 00:01 2024-02-14T17:34:07.699 INFO:teuthology.orchestra.run.smithi178.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 257 kB/s | 17 kB 00:00 2024-02-14T17:34:07.749 INFO:teuthology.orchestra.run.smithi184.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1430.g8727 4.9 MB/s | 247 kB 00:00 2024-02-14T17:34:07.767 INFO:teuthology.orchestra.run.smithi178.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 512 kB/s | 34 kB 00:00 2024-02-14T17:34:07.909 INFO:teuthology.orchestra.run.smithi178.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.7 MB/s | 3.0 MB 00:00 2024-02-14T17:34:08.066 INFO:teuthology.orchestra.run.smithi184.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 590 kB/s | 187 kB 00:00 2024-02-14T17:34:08.224 INFO:teuthology.orchestra.run.smithi184.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 912 kB/s | 144 kB 00:00 2024-02-14T17:34:08.300 INFO:teuthology.orchestra.run.smithi184.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 420 kB/s | 32 kB 00:00 2024-02-14T17:34:08.358 INFO:teuthology.orchestra.run.smithi184.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 816 kB/s | 47 kB 00:00 2024-02-14T17:34:08.384 INFO:teuthology.orchestra.run.smithi178.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 7.7 MB/s | 4.7 MB 00:00 2024-02-14T17:34:08.434 INFO:teuthology.orchestra.run.smithi184.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 516 kB/s | 39 kB 00:00 2024-02-14T17:34:08.443 INFO:teuthology.orchestra.run.smithi178.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.0 MB/s | 4.8 MB 00:00 2024-02-14T17:34:08.651 INFO:teuthology.orchestra.run.smithi178.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.0 MB/s | 217 kB 00:00 2024-02-14T17:34:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 241 kB 00:00 2024-02-14T17:34:08.826 INFO:teuthology.orchestra.run.smithi178.stdout:(8/12): ceph-mgr-diskprediction-local-19.0.0-14 5.1 MB/s | 7.4 MB 00:01 2024-02-14T17:34:08.917 INFO:teuthology.orchestra.run.smithi184.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 4.6 MB/s | 1.1 MB 00:00 2024-02-14T17:34:08.968 INFO:teuthology.orchestra.run.smithi184.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-02-14T17:34:09.051 INFO:teuthology.orchestra.run.smithi184.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 1.5 MB/s | 124 kB 00:00 2024-02-14T17:34:09.244 INFO:teuthology.orchestra.run.smithi184.stdout:(14/36): python3-requests-2.25.1-8.el9.noarch.r 590 kB/s | 113 kB 00:00 2024-02-14T17:34:09.251 INFO:teuthology.orchestra.run.smithi178.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 458 kB/s | 194 kB 00:00 2024-02-14T17:34:09.295 INFO:teuthology.orchestra.run.smithi184.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-02-14T17:34:09.402 INFO:teuthology.orchestra.run.smithi178.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-02-14T17:34:09.435 INFO:teuthology.orchestra.run.smithi178.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 802 kB 00:00 2024-02-14T17:34:09.461 INFO:teuthology.orchestra.run.smithi184.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-02-14T17:34:09.811 INFO:teuthology.orchestra.run.smithi184.stdout:(17/36): ceph-mgr-19.0.0-1430.g8727ce93.el9.x86 401 kB/s | 1.5 MB 00:03 2024-02-14T17:34:09.853 INFO:teuthology.orchestra.run.smithi184.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 385 kB/s | 151 kB 00:00 2024-02-14T17:34:09.912 INFO:teuthology.orchestra.run.smithi178.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 373 kB/s | 245 kB 00:00 2024-02-14T17:34:09.913 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:09.913 INFO:teuthology.orchestra.run.smithi178.stdout:Total 13 MB/s | 37 MB 00:02 2024-02-14T17:34:09.926 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:34:09.950 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:34:09.950 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:34:09.961 INFO:teuthology.orchestra.run.smithi184.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-02-14T17:34:09.987 INFO:teuthology.orchestra.run.smithi184.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 247 kB/s | 43 kB 00:00 2024-02-14T17:34:10.062 INFO:teuthology.orchestra.run.smithi184.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 3.5 MB/s | 358 kB 00:00 2024-02-14T17:34:10.137 INFO:teuthology.orchestra.run.smithi184.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 71 kB/s | 11 kB 00:00 2024-02-14T17:34:10.162 INFO:teuthology.orchestra.run.smithi184.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 176 kB/s | 18 kB 00:00 2024-02-14T17:34:10.213 INFO:teuthology.orchestra.run.smithi184.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-02-14T17:34:10.255 INFO:teuthology.orchestra.run.smithi184.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 212 kB/s | 19 kB 00:00 2024-02-14T17:34:10.296 INFO:teuthology.orchestra.run.smithi184.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 235 kB/s | 20 kB 00:00 2024-02-14T17:34:10.305 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:34:10.305 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:34:10.330 INFO:teuthology.orchestra.run.smithi184.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 617 kB/s | 46 kB 00:00 2024-02-14T17:34:10.381 INFO:teuthology.orchestra.run.smithi184.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 943 kB/s | 79 kB 00:00 2024-02-14T17:34:10.406 INFO:teuthology.orchestra.run.smithi184.stdout:(29/36): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-02-14T17:34:10.448 INFO:teuthology.orchestra.run.smithi184.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 245 kB/s | 16 kB 00:00 2024-02-14T17:34:10.473 INFO:teuthology.orchestra.run.smithi184.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-02-14T17:34:10.515 INFO:teuthology.orchestra.run.smithi184.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 535 kB/s | 36 kB 00:00 2024-02-14T17:34:10.557 INFO:teuthology.orchestra.run.smithi184.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-02-14T17:34:10.632 INFO:teuthology.orchestra.run.smithi184.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 267 kB/s | 20 kB 00:00 2024-02-14T17:34:10.732 INFO:teuthology.orchestra.run.smithi184.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.9 MB/s | 427 kB 00:00 2024-02-14T17:34:11.008 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:34:11.083 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-14T17:34:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-14T17:34:11.215 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-14T17:34:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-14T17:34:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-14T17:34:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-14T17:34:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-14T17:34:12.222 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-14T17:34:12.346 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-14T17:34:12.624 INFO:teuthology.orchestra.run.smithi184.stdout:(36/36): ceph-mon-19.0.0-1430.g8727ce93.el9.x86 757 kB/s | 4.7 MB 00:06 2024-02-14T17:34:12.628 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:12.628 INFO:teuthology.orchestra.run.smithi184.stdout:Total 4.3 MB/s | 30 MB 00:06 2024-02-14T17:34:12.646 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:34:12.690 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:34:12.690 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:34:13.116 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:34:13.117 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:34:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-14T17:34:14.513 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:34:14.568 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-14T17:34:14.607 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-14T17:34:14.635 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-14T17:34:14.637 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-14T17:34:14.657 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 12/12 2024-02-14T17:34:14.708 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-14T17:34:14.744 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-14T17:34:14.777 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-14T17:34:14.869 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-14T17:34:14.956 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-14T17:34:14.996 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-14T17:34:15.068 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-14T17:34:15.130 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-14T17:34:15.169 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-14T17:34:15.227 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-14T17:34:15.282 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-14T17:34:15.468 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-14T17:34:15.515 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-14T17:34:15.554 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-14T17:34:15.580 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-14T17:34:15.609 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-14T17:34:15.686 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-14T17:34:15.720 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-14T17:34:15.753 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-14T17:34:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-14T17:34:15.828 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-14T17:34:15.872 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-14T17:34:15.961 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-14T17:34:15.994 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-14T17:34:16.067 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-14T17:34:16.204 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-14T17:34:16.329 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-14T17:34:16.487 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9. 31/36 2024-02-14T17:34:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 32/36 2024-02-14T17:34:16.541 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 32/36 2024-02-14T17:34:16.541 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:34:16.541 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-14T17:34:16.542 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T17:34:16.542 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T17:34:16.542 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:17.192 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 12/12 2024-02-14T17:34:17.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 1/12 2024-02-14T17:34:17.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-14T17:34:17.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-14T17:34:17.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-14T17:34:17.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-14T17:34:17.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-14T17:34:17.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-14T17:34:17.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-14T17:34:17.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-14T17:34:17.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-14T17:34:17.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-14T17:34:17.672 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-14T17:34:17.672 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:17.672 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:34:17.673 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:17.673 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-14T17:34:17.673 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-14T17:34:17.673 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-14T17:34:17.673 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-14T17:34:17.673 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-14T17:34:17.674 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-14T17:34:17.674 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-14T17:34:17.674 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-14T17:34:17.674 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-14T17:34:17.674 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-14T17:34:17.674 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-14T17:34:17.674 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:17.675 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:17.960 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 33/36 2024-02-14T17:34:17.984 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 33/36 2024-02-14T17:34:17.984 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:34:17.984 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-14T17:34:17.984 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T17:34:17.985 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T17:34:17.985 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:18.037 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-rook 2024-02-14T17:34:18.425 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 34/36 2024-02-14T17:34:18.452 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 34/36 2024-02-14T17:34:18.452 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:34:18.452 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-14T17:34:18.452 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T17:34:18.452 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T17:34:18.452 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:18.627 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:20 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:18.800 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:18.801 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================== 2024-02-14T17:34:18.801 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-02-14T17:34:18.801 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================== 2024-02-14T17:34:18.802 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:34:18.802 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 52 k 2024-02-14T17:34:18.802 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-02-14T17:34:18.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-14T17:34:18.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-14T17:34:18.802 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-14T17:34:18.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-14T17:34:18.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-14T17:34:18.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-14T17:34:18.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-14T17:34:18.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-14T17:34:18.803 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-14T17:34:18.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-14T17:34:18.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-14T17:34:18.804 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-14T17:34:18.804 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:18.804 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:34:18.804 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================== 2024-02-14T17:34:18.804 INFO:teuthology.orchestra.run.smithi178.stdout:Install 13 Packages 2024-02-14T17:34:18.805 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:18.806 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 2.0 M 2024-02-14T17:34:18.806 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 26 M 2024-02-14T17:34:18.806 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:34:18.814 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 35/36 2024-02-14T17:34:18.832 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 35/36 2024-02-14T17:34:18.832 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:34:18.832 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-14T17:34:18.832 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T17:34:18.833 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T17:34:18.833 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:18.898 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 36/36 2024-02-14T17:34:19.149 INFO:teuthology.orchestra.run.smithi178.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 151 kB/s | 24 kB 00:00 2024-02-14T17:34:19.175 INFO:teuthology.orchestra.run.smithi178.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 97 kB/s | 18 kB 00:00 2024-02-14T17:34:19.283 INFO:teuthology.orchestra.run.smithi178.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 174 kB 00:00 2024-02-14T17:34:19.309 INFO:teuthology.orchestra.run.smithi178.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 998 kB/s | 132 kB 00:00 2024-02-14T17:34:19.351 INFO:teuthology.orchestra.run.smithi178.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.1 MB/s | 211 kB 00:00 2024-02-14T17:34:19.376 INFO:teuthology.orchestra.run.smithi178.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 637 kB/s | 43 kB 00:00 2024-02-14T17:34:19.410 INFO:teuthology.orchestra.run.smithi178.stdout:(7/13): ceph-mgr-rook-19.0.0-1430.g8727ce93.el9 124 kB/s | 52 kB 00:00 2024-02-14T17:34:19.660 INFO:teuthology.orchestra.run.smithi178.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 104 kB/s | 32 kB 00:00 2024-02-14T17:34:19.685 INFO:teuthology.orchestra.run.smithi178.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 48 kB/s | 15 kB 00:00 2024-02-14T17:34:19.894 INFO:teuthology.orchestra.run.smithi178.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 284 kB/s | 59 kB 00:00 2024-02-14T17:34:19.960 INFO:teuthology.orchestra.run.smithi178.stdout:(11/13): python3-google-auth-2.27.0-1.el9.noarc 368 kB/s | 203 kB 00:00 2024-02-14T17:34:20.094 INFO:teuthology.orchestra.run.smithi178.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 448 kB/s | 90 kB 00:00 2024-02-14T17:34:20.294 INFO:teuthology.orchestra.run.smithi178.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.6 MB/s | 1.0 MB 00:00 2024-02-14T17:34:20.296 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:20.297 INFO:teuthology.orchestra.run.smithi178.stdout:Total 1.4 MB/s | 2.0 MB 00:01 2024-02-14T17:34:20.297 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:34:20.312 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:34:20.312 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:34:20.409 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 36/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 1/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 2/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 3/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 4/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 5/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9. 6/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-14T17:34:20.410 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-14T17:34:20.411 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-14T17:34:20.413 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-14T17:34:20.413 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-14T17:34:20.413 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-14T17:34:20.413 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-14T17:34:20.413 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-14T17:34:20.413 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-14T17:34:20.413 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-14T17:34:20.416 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:34:20.419 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:34:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:34:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-14T17:34:21.164 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-14T17:34:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-14T17:34:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-14T17:34:21.296 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-14T17:34:21.307 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-14T17:34:21.308 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-14T17:34:21.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-14T17:34:21.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-14T17:34:21.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-14T17:34:21.309 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-14T17:34:21.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-14T17:34:21.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-14T17:34:21.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-14T17:34:21.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-14T17:34:21.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-14T17:34:21.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-14T17:34:21.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-14T17:34:21.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-14T17:34:21.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-14T17:34:21.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-02-14T17:34:21.311 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-14T17:34:21.312 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-14T17:34:21.312 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-14T17:34:21.312 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-14T17:34:21.312 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-14T17:34:21.312 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-14T17:34:21.312 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-14T17:34:21.312 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-14T17:34:21.312 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-14T17:34:21.312 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-14T17:34:21.316 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-14T17:34:21.316 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:21.316 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:34:21.389 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-14T17:34:21.496 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-14T17:34:21.546 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-14T17:34:21.694 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-base 2024-02-14T17:34:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-14T17:34:22.273 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:22 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:34:22.311 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-14T17:34:22.336 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-14T17:34:22.376 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-14T17:34:22.387 INFO:teuthology.orchestra.run.smithi184.stdout:Package ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:34:22.398 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 13/13 2024-02-14T17:34:22.438 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:34:22.439 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-14T17:34:22.439 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:34:22.503 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install cephadm 2024-02-14T17:34:23.073 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:23 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:34:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 13/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 1/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-14T17:34:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-14T17:34:23.239 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 748 k 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 748 k 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 754 k 2024-02-14T17:34:23.240 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:34:23.532 INFO:teuthology.orchestra.run.smithi184.stdout:cephadm-19.0.0-1430.g8727ce93.el9.noarch.rpm 2.5 MB/s | 748 kB 00:00 2024-02-14T17:34:23.532 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:23.533 INFO:teuthology.orchestra.run.smithi184.stdout:Total 2.5 MB/s | 748 kB 00:00 2024-02-14T17:34:23.533 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:34:23.538 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:34:23.538 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:34:23.544 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:34:23.544 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:34:23.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-14T17:34:23.574 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:23.575 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:34:23.575 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:23.575 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-14T17:34:23.575 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-14T17:34:23.575 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-14T17:34:23.575 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-14T17:34:23.575 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-14T17:34:23.575 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-14T17:34:23.576 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-14T17:34:23.576 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-14T17:34:23.576 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-14T17:34:23.576 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-14T17:34:23.576 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-14T17:34:23.576 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-14T17:34:23.576 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:23.576 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:23.619 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:34:23.837 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-cephadm 2024-02-14T17:34:24.037 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:34:24.128 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:34:24.410 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:26 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:24.574 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:24.575 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================== 2024-02-14T17:34:24.575 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-02-14T17:34:24.575 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================== 2024-02-14T17:34:24.575 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 138 k 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================== 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout:Install 6 Packages 2024-02-14T17:34:24.576 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:24.577 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 6.8 M 2024-02-14T17:34:24.577 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 33 M 2024-02-14T17:34:24.577 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:34:24.584 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:34:24.907 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:34:24.907 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:24.907 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:34:24.907 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:24.908 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:24.908 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:34:25.036 INFO:teuthology.orchestra.run.smithi178.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 716 kB/s | 227 kB 00:00 2024-02-14T17:34:25.108 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-immutable-object-cache 2024-02-14T17:34:25.246 INFO:teuthology.orchestra.run.smithi178.stdout:(2/6): ceph-mgr-cephadm-19.0.0-1430.g8727ce93.e 263 kB/s | 138 kB 00:00 2024-02-14T17:34:25.321 INFO:teuthology.orchestra.run.smithi178.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 9.6 MB/s | 5.8 MB 00:00 2024-02-14T17:34:25.690 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:25 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:34:25.787 INFO:teuthology.orchestra.run.smithi178.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 107 kB/s | 58 kB 00:00 2024-02-14T17:34:25.854 INFO:teuthology.orchestra.run.smithi178.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 96 kB/s | 51 kB 00:00 2024-02-14T17:34:25.860 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:34:25.860 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 145 k 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 145 k 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 436 k 2024-02-14T17:34:25.861 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:34:25.937 INFO:teuthology.orchestra.run.smithi178.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 608 kB/s | 548 kB 00:00 2024-02-14T17:34:25.941 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:25.942 INFO:teuthology.orchestra.run.smithi178.stdout:Total 5.0 MB/s | 6.8 MB 00:01 2024-02-14T17:34:25.942 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:34:25.951 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:34:25.951 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:34:26.083 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-immutable-object-cache-19.0.0-1430.g8727ce 668 kB/s | 145 kB 00:00 2024-02-14T17:34:26.083 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:26.083 INFO:teuthology.orchestra.run.smithi184.stdout:Total 663 kB/s | 145 kB 00:00 2024-02-14T17:34:26.083 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:34:26.087 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:34:26.087 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:34:26.140 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:34:26.141 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:34:26.280 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:34:26.330 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:34:26.353 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:34:26.353 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:34:26.353 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-14T17:34:26.354 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:34:26.526 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-14T17:34:26.572 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-14T17:34:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-14T17:34:27.165 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-14T17:34:27.227 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-14T17:34:27.260 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 6/6 2024-02-14T17:34:27.377 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:34:27.378 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:27.378 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:34:27.378 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:27.378 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:27.378 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:34:27.620 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr 2024-02-14T17:34:28.109 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 6/6 2024-02-14T17:34:28.110 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/6 2024-02-14T17:34:28.110 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-14T17:34:28.110 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-14T17:34:28.110 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-14T17:34:28.110 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-14T17:34:28.206 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:28 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:34:28.320 INFO:teuthology.orchestra.run.smithi184.stdout:Package ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:34:28.371 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:34:28.372 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-14T17:34:28.372 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:34:28.439 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-dashboard 2024-02-14T17:34:28.443 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-14T17:34:28.443 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:28.443 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:34:28.443 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:28.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-14T17:34:28.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-14T17:34:28.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-14T17:34:28.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-14T17:34:28.444 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-14T17:34:28.444 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:28.444 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:28.725 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-fuse 2024-02-14T17:34:29.016 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:29 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:34:29.184 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:34:29.185 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-02-14T17:34:29.185 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-14T17:34:29.185 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-02-14T17:34:29.185 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:34:29.185 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 1.8 M 2024-02-14T17:34:29.185 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-14T17:34:29.185 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-14T17:34:29.185 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 31 k 2024-02-14T17:34:29.185 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 15 k 2024-02-14T17:34:29.185 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout:Install 12 Packages 2024-02-14T17:34:29.186 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:29.187 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 6.9 M 2024-02-14T17:34:29.187 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 26 M 2024-02-14T17:34:29.187 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:34:29.302 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:31 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:29.472 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:29.473 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:29.474 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-02-14T17:34:29.474 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:29.474 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:34:29.474 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 886 k 2024-02-14T17:34:29.474 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-02-14T17:34:29.474 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-14T17:34:29.474 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:29.474 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:34:29.474 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:29.474 INFO:teuthology.orchestra.run.smithi178.stdout:Install 2 Packages 2024-02-14T17:34:29.475 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:29.475 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 965 k 2024-02-14T17:34:29.475 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 2.8 M 2024-02-14T17:34:29.475 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:34:29.674 INFO:teuthology.orchestra.run.smithi178.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 393 kB/s | 79 kB 00:00 2024-02-14T17:34:29.791 INFO:teuthology.orchestra.run.smithi184.stdout:(1/12): ceph-grafana-dashboards-19.0.0-1430.g87 91 kB/s | 31 kB 00:00 2024-02-14T17:34:29.825 INFO:teuthology.orchestra.run.smithi184.stdout:(2/12): ceph-prometheus-alerts-19.0.0-1430.g872 40 kB/s | 15 kB 00:00 2024-02-14T17:34:30.033 INFO:teuthology.orchestra.run.smithi184.stdout:(3/12): ceph-mgr-dashboard-19.0.0-1430.g8727ce9 3.0 MB/s | 1.8 MB 00:00 2024-02-14T17:34:30.159 INFO:teuthology.orchestra.run.smithi184.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 766 kB/s | 238 kB 00:00 2024-02-14T17:34:30.226 INFO:teuthology.orchestra.run.smithi184.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.3 MB/s | 1.0 MB 00:00 2024-02-14T17:34:30.326 INFO:teuthology.orchestra.run.smithi184.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-02-14T17:34:30.410 INFO:teuthology.orchestra.run.smithi184.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.1 MB/s | 548 kB 00:00 2024-02-14T17:34:30.463 INFO:teuthology.orchestra.run.smithi184.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-02-14T17:34:30.518 INFO:teuthology.orchestra.run.smithi184.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 526 kB/s | 31 kB 00:00 2024-02-14T17:34:30.577 INFO:teuthology.orchestra.run.smithi184.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-02-14T17:34:30.627 INFO:teuthology.orchestra.run.smithi184.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 6.8 MB/s | 2.0 MB 00:00 2024-02-14T17:34:30.641 INFO:teuthology.orchestra.run.smithi178.stdout:(2/2): ceph-fuse-19.0.0-1430.g8727ce93.el9.x86_ 760 kB/s | 886 kB 00:01 2024-02-14T17:34:30.642 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:30.643 INFO:teuthology.orchestra.run.smithi178.stdout:Total 825 kB/s | 965 kB 00:01 2024-02-14T17:34:30.643 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:34:30.666 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:34:30.666 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:34:30.747 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:34:30.747 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:34:30.932 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:34:31.066 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-14T17:34:31.140 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2/2 2024-02-14T17:34:31.361 INFO:teuthology.orchestra.run.smithi184.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 651 kB/s | 863 kB 00:01 2024-02-14T17:34:31.366 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:31.367 INFO:teuthology.orchestra.run.smithi184.stdout:Total 3.1 MB/s | 6.9 MB 00:02 2024-02-14T17:34:31.386 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:34:31.404 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:34:31.404 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:34:31.581 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:34:31.582 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:34:31.806 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2/2 2024-02-14T17:34:31.807 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/2 2024-02-14T17:34:32.105 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:34:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-14T17:34:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:32.142 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:34:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-14T17:34:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:32.142 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:32.170 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-14T17:34:32.204 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-14T17:34:32.251 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-14T17:34:32.282 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-14T17:34:32.375 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-14T17:34:32.450 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-volume 2024-02-14T17:34:32.499 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-14T17:34:32.584 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-14T17:34:32.628 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-14T17:34:32.656 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-14T17:34:32.680 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9 10/12 2024-02-14T17:34:32.910 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el 11/12 2024-02-14T17:34:32.946 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 12/12 2024-02-14T17:34:33.045 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:35 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:33.213 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:33.214 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:33.214 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-02-14T17:34:33.214 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:33.214 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:34:33.214 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 264 k 2024-02-14T17:34:33.214 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-02-14T17:34:33.214 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-14T17:34:33.214 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:33.214 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:34:33.214 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:33.215 INFO:teuthology.orchestra.run.smithi178.stdout:Install 2 Packages 2024-02-14T17:34:33.215 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:33.215 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 332 k 2024-02-14T17:34:33.215 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 1.5 M 2024-02-14T17:34:33.215 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:34:33.466 INFO:teuthology.orchestra.run.smithi178.stdout:(1/2): ceph-volume-19.0.0-1430.g8727ce93.el9.no 1.0 MB/s | 264 kB 00:00 2024-02-14T17:34:33.487 INFO:teuthology.orchestra.run.smithi178.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 253 kB/s | 69 kB 00:00 2024-02-14T17:34:33.488 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:33.488 INFO:teuthology.orchestra.run.smithi178.stdout:Total 1.2 MB/s | 332 kB 00:00 2024-02-14T17:34:33.488 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:34:33.497 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:34:33.498 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:34:33.542 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:34:33.542 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:34:33.709 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:34:33.812 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 12/12 2024-02-14T17:34:33.812 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el 1/12 2024-02-14T17:34:33.812 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 2/12 2024-02-14T17:34:33.812 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9 3/12 2024-02-14T17:34:33.812 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-14T17:34:33.812 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-14T17:34:33.813 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-14T17:34:33.813 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-14T17:34:33.813 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-14T17:34:33.813 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-14T17:34:33.813 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-14T17:34:33.813 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-14T17:34:33.885 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-14T17:34:33.923 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2/2 2024-02-14T17:34:33.938 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2/2 2024-02-14T17:34:33.938 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:34:33.938 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-14T17:34:33.938 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:34.199 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-14T17:34:34.200 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-14T17:34:34.201 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-14T17:34:34.201 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-14T17:34:34.201 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:34.201 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:34:34.453 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-14T17:34:34.746 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 1/2 2024-02-14T17:34:35.031 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:35 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:34:35.125 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-14T17:34:35.125 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:35.126 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:34:35.126 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:35.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-14T17:34:35.126 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:35.126 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:35.199 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:34:35.200 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-02-14T17:34:35.200 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-14T17:34:35.200 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-02-14T17:34:35.200 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:34:35.200 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 7.4 M 2024-02-14T17:34:35.200 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-14T17:34:35.200 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-14T17:34:35.200 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout:Install 12 Packages 2024-02-14T17:34:35.201 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:35.202 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 37 M 2024-02-14T17:34:35.202 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 209 M 2024-02-14T17:34:35.202 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:34:35.428 INFO:teuthology.orchestra.run.smithi184.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 174 kB/s | 32 kB 00:00 2024-02-14T17:34:35.476 INFO:teuthology.orchestra.run.smithi184.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 344 kB/s | 17 kB 00:00 2024-02-14T17:34:35.499 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librados-devel 2024-02-14T17:34:35.526 INFO:teuthology.orchestra.run.smithi184.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-02-14T17:34:35.677 INFO:teuthology.orchestra.run.smithi184.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.9 MB/s | 3.0 MB 00:00 2024-02-14T17:34:36.079 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:38 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:36.119 INFO:teuthology.orchestra.run.smithi184.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 8.0 MB/s | 4.7 MB 00:00 2024-02-14T17:34:36.194 INFO:teuthology.orchestra.run.smithi184.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.3 MB/s | 4.8 MB 00:00 2024-02-14T17:34:36.219 INFO:teuthology.orchestra.run.smithi184.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.2 MB/s | 217 kB 00:00 2024-02-14T17:34:36.245 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:36.246 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-02-14T17:34:36.246 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:36.246 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:34:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 127 k 2024-02-14T17:34:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:36.246 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:34:36.246 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:36.247 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-02-14T17:34:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:36.247 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 127 k 2024-02-14T17:34:36.247 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 456 k 2024-02-14T17:34:36.247 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:34:36.311 INFO:teuthology.orchestra.run.smithi184.stdout:(8/12): ceph-mgr-diskprediction-local-19.0.0-14 6.9 MB/s | 7.4 MB 00:01 2024-02-14T17:34:36.553 INFO:teuthology.orchestra.run.smithi184.stdout:(9/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 802 kB 00:00 2024-02-14T17:34:36.620 INFO:teuthology.orchestra.run.smithi184.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 632 kB/s | 194 kB 00:00 2024-02-14T17:34:36.734 INFO:teuthology.orchestra.run.smithi178.stdout:librados-devel-19.0.0-1430.g8727ce93.el9.x86_64 260 kB/s | 127 kB 00:00 2024-02-14T17:34:36.734 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:36.735 INFO:teuthology.orchestra.run.smithi178.stdout:Total 259 kB/s | 127 kB 00:00 2024-02-14T17:34:36.735 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:34:36.744 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:34:36.744 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:34:36.796 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:34:36.796 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:34:36.879 INFO:teuthology.orchestra.run.smithi184.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-02-14T17:34:36.980 INFO:teuthology.orchestra.run.smithi184.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 579 kB/s | 245 kB 00:00 2024-02-14T17:34:36.981 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:36.981 INFO:teuthology.orchestra.run.smithi184.stdout:Total 21 MB/s | 37 MB 00:01 2024-02-14T17:34:36.994 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:34:37.007 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:34:37.017 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:34:37.018 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:34:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:37.365 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:34:37.366 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:34:37.558 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:37.897 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:37.897 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:37.897 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:34:37.897 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:37.897 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:37.897 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:38.102 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:34:38.127 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install libcephfs2 2024-02-14T17:34:38.168 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-14T17:34:38.200 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-14T17:34:38.293 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-14T17:34:38.336 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-14T17:34:38.512 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-14T17:34:38.543 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-14T17:34:38.627 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-14T17:34:38.714 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:40 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:38.829 INFO:teuthology.orchestra.run.smithi178.stdout:Package libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:34:38.880 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:38.881 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-02-14T17:34:38.881 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:38.951 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install libcephfs-devel 2024-02-14T17:34:39.290 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-14T17:34:39.398 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-14T17:34:39.531 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:41 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:39.696 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:39.697 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:39.697 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-02-14T17:34:39.697 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:39.697 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:34:39.697 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 31 k 2024-02-14T17:34:39.697 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:39.697 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:34:39.697 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:39.697 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-02-14T17:34:39.697 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:39.698 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 31 k 2024-02-14T17:34:39.698 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 140 k 2024-02-14T17:34:39.698 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:34:39.862 INFO:teuthology.orchestra.run.smithi178.stdout:libcephfs-devel-19.0.0-1430.g8727ce93.el9.x86_6 190 kB/s | 31 kB 00:00 2024-02-14T17:34:39.863 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:39.863 INFO:teuthology.orchestra.run.smithi178.stdout:Total 189 kB/s | 31 kB 00:00 2024-02-14T17:34:39.863 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:34:39.866 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:34:39.866 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:34:39.885 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:34:39.885 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:34:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:34:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:40.478 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:40.626 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-14T17:34:40.760 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:40.760 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:40.760 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:34:40.760 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:40.760 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:40.760 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:40.953 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librados2 2024-02-14T17:34:41.528 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:43 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:41.641 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:34:41.656 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-14T17:34:41.692 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:41.693 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-02-14T17:34:41.693 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:41.693 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 12/12 2024-02-14T17:34:41.762 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librbd1 2024-02-14T17:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:44 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:42.453 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:34:42.505 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:42.506 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-02-14T17:34:42.506 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:42.580 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rados 2024-02-14T17:34:43.154 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:45 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:43.269 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:34:43.319 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:43.320 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-02-14T17:34:43.321 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:43.391 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rgw 2024-02-14T17:34:43.966 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:45 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:44.080 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:34:44.131 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:44.132 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-02-14T17:34:44.132 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:44.197 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-cephfs 2024-02-14T17:34:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 12/12 2024-02-14T17:34:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 1/12 2024-02-14T17:34:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-14T17:34:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-14T17:34:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-14T17:34:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-14T17:34:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-14T17:34:44.254 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-14T17:34:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-14T17:34:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-14T17:34:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-14T17:34:44.255 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-14T17:34:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-14T17:34:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:44.741 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:34:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-14T17:34:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-14T17:34:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-14T17:34:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-14T17:34:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-14T17:34:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-14T17:34:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-14T17:34:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-14T17:34:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-14T17:34:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-14T17:34:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-14T17:34:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:44.744 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:34:44.770 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:46 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:44.883 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:34:44.933 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:44.934 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-02-14T17:34:44.935 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:45.004 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rbd 2024-02-14T17:34:45.111 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-rook 2024-02-14T17:34:45.616 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:47 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:45.691 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:45 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:34:45.736 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:34:45.789 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:45.790 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-02-14T17:34:45.790 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:45.860 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:34:45.861 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 52 k 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-14T17:34:45.862 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-14T17:34:45.863 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-14T17:34:45.863 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-14T17:34:45.863 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-14T17:34:45.863 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-14T17:34:45.863 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:45.863 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:34:45.863 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-14T17:34:45.863 INFO:teuthology.orchestra.run.smithi184.stdout:Install 13 Packages 2024-02-14T17:34:45.863 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:45.864 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 2.0 M 2024-02-14T17:34:45.864 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 26 M 2024-02-14T17:34:45.864 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:34:45.865 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-fuse 2024-02-14T17:34:46.150 INFO:teuthology.orchestra.run.smithi184.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 145 kB/s | 24 kB 00:00 2024-02-14T17:34:46.176 INFO:teuthology.orchestra.run.smithi184.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 94 kB/s | 18 kB 00:00 2024-02-14T17:34:46.276 INFO:teuthology.orchestra.run.smithi184.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 174 kB 00:00 2024-02-14T17:34:46.301 INFO:teuthology.orchestra.run.smithi184.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 132 kB 00:00 2024-02-14T17:34:46.343 INFO:teuthology.orchestra.run.smithi184.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.1 MB/s | 211 kB 00:00 2024-02-14T17:34:46.368 INFO:teuthology.orchestra.run.smithi184.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 638 kB/s | 43 kB 00:00 2024-02-14T17:34:46.394 INFO:teuthology.orchestra.run.smithi184.stdout:(7/13): ceph-mgr-rook-19.0.0-1430.g8727ce93.el9 127 kB/s | 52 kB 00:00 2024-02-14T17:34:46.461 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:48 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:46.602 INFO:teuthology.orchestra.run.smithi184.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 124 kB/s | 32 kB 00:00 2024-02-14T17:34:46.625 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:46.626 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:46.626 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-02-14T17:34:46.626 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:46.626 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:34:46.626 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 85 k 2024-02-14T17:34:46.626 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:46.626 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:34:46.626 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:46.626 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-02-14T17:34:46.627 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:46.627 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 85 k 2024-02-14T17:34:46.627 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 231 k 2024-02-14T17:34:46.627 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:34:46.628 INFO:teuthology.orchestra.run.smithi184.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 58 kB/s | 15 kB 00:00 2024-02-14T17:34:46.719 INFO:teuthology.orchestra.run.smithi184.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 645 kB/s | 59 kB 00:00 2024-02-14T17:34:46.778 INFO:teuthology.orchestra.run.smithi184.stdout:(11/13): python3-google-auth-2.27.0-1.el9.noarc 528 kB/s | 203 kB 00:00 2024-02-14T17:34:46.803 INFO:teuthology.orchestra.run.smithi184.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.1 MB/s | 90 kB 00:00 2024-02-14T17:34:47.098 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-fuse-19.0.0-1430.g8727ce93.el9.x86_64.rpm 181 kB/s | 85 kB 00:00 2024-02-14T17:34:47.099 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:47.099 INFO:teuthology.orchestra.run.smithi178.stdout:Total 180 kB/s | 85 kB 00:00 2024-02-14T17:34:47.099 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:34:47.106 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:34:47.106 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:34:47.161 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:34:47.161 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:34:47.262 INFO:teuthology.orchestra.run.smithi184.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.5 MB/s | 1.0 MB 00:00 2024-02-14T17:34:47.265 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:47.266 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2024-02-14T17:34:47.266 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:34:47.279 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:34:47.279 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:34:47.331 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:34:47.376 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:34:47.377 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:34:47.429 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:47.864 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:47.908 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:34:48.065 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-14T17:34:48.121 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-14T17:34:48.193 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-14T17:34:48.229 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:48.229 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:48.229 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:34:48.230 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:48.230 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:48.230 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:48.230 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-14T17:34:48.280 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-14T17:34:48.381 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-14T17:34:48.478 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-mirror 2024-02-14T17:34:48.490 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-14T17:34:48.539 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-14T17:34:49.065 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:51 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:49.244 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:49.245 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:49.245 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-02-14T17:34:49.245 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:49.245 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:34:49.245 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.1 M 2024-02-14T17:34:49.245 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:49.245 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:34:49.245 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:49.245 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-02-14T17:34:49.245 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:49.246 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 3.1 M 2024-02-14T17:34:49.246 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 13 M 2024-02-14T17:34:49.246 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:34:49.258 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-14T17:34:49.320 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-14T17:34:49.346 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-14T17:34:49.395 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-14T17:34:49.424 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 13/13 2024-02-14T17:34:49.826 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-mirror-19.0.0-1430.g8727ce93.el9.x86_64.rpm 5.3 MB/s | 3.1 MB 00:00 2024-02-14T17:34:49.827 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:49.827 INFO:teuthology.orchestra.run.smithi178.stdout:Total 5.3 MB/s | 3.1 MB 00:00 2024-02-14T17:34:49.827 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:34:49.836 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:34:49.836 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:34:49.910 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:34:49.910 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:34:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 13/13 2024-02-14T17:34:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 1/13 2024-02-14T17:34:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-14T17:34:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-14T17:34:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-14T17:34:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-14T17:34:50.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-14T17:34:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-14T17:34:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-14T17:34:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-14T17:34:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-14T17:34:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-14T17:34:50.227 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-14T17:34:50.340 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:34:50.410 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:50.434 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:50.434 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:34:50.435 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-14T17:34:50.435 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T17:34:50.435 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T17:34:50.435 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:50.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-14T17:34:50.598 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:50.598 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:34:50.598 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:50.598 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-14T17:34:50.598 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-14T17:34:50.598 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-14T17:34:50.598 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-14T17:34:50.598 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-14T17:34:50.598 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-14T17:34:50.598 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-14T17:34:50.599 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-14T17:34:50.599 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-14T17:34:50.599 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-14T17:34:50.599 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-14T17:34:50.599 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-14T17:34:50.599 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:50.599 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:34:50.846 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-cephadm 2024-02-14T17:34:51.421 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:51 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:34:51.587 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:34:51.588 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-14T17:34:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-14T17:34:51.588 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-14T17:34:51.588 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:34:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 138 k 2024-02-14T17:34:51.588 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-14T17:34:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-14T17:34:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-14T17:34:51.588 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-14T17:34:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-14T17:34:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-14T17:34:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:51.589 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:34:51.589 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-14T17:34:51.589 INFO:teuthology.orchestra.run.smithi184.stdout:Install 6 Packages 2024-02-14T17:34:51.589 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:51.589 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 6.8 M 2024-02-14T17:34:51.590 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 33 M 2024-02-14T17:34:51.590 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:34:51.619 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:51.619 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:51.619 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:34:51.619 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:51.619 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:51.619 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:51.813 INFO:teuthology.orchestra.run.smithi184.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 227 kB 00:00 2024-02-14T17:34:51.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-nbd 2024-02-14T17:34:51.896 INFO:teuthology.orchestra.run.smithi184.stdout:(2/6): ceph-mgr-cephadm-19.0.0-1430.g8727ce93.e 537 kB/s | 138 kB 00:00 2024-02-14T17:34:52.074 INFO:teuthology.orchestra.run.smithi184.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 5.8 MB 00:00 2024-02-14T17:34:52.299 INFO:teuthology.orchestra.run.smithi184.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 144 kB/s | 58 kB 00:00 2024-02-14T17:34:52.416 INFO:teuthology.orchestra.run.smithi184.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 149 kB/s | 51 kB 00:00 2024-02-14T17:34:52.475 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:54 ago on Wed 14 Feb 2024 05:32:58 PM UTC. 2024-02-14T17:34:52.525 INFO:teuthology.orchestra.run.smithi184.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 771 kB/s | 548 kB 00:00 2024-02-14T17:34:52.527 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:52.527 INFO:teuthology.orchestra.run.smithi184.stdout:Total 7.2 MB/s | 6.8 MB 00:00 2024-02-14T17:34:52.527 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:34:52.537 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:34:52.537 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:34:52.630 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:34:52.630 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:34:52.639 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-02-14T17:34:52.640 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:52.640 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-02-14T17:34:52.640 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:52.640 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-02-14T17:34:52.640 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 171 k 2024-02-14T17:34:52.640 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:52.640 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-02-14T17:34:52.640 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-02-14T17:34:52.640 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-02-14T17:34:52.640 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:52.641 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 171 k 2024-02-14T17:34:52.641 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 494 k 2024-02-14T17:34:52.641 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-02-14T17:34:52.871 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-nbd-19.0.0-1430.g8727ce93.el9.x86_64.rpm 745 kB/s | 171 kB 00:00 2024-02-14T17:34:52.871 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:52.871 INFO:teuthology.orchestra.run.smithi178.stdout:Total 740 kB/s | 171 kB 00:00 2024-02-14T17:34:52.872 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-02-14T17:34:52.879 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-02-14T17:34:52.879 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-02-14T17:34:52.926 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:34:52.935 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-02-14T17:34:52.935 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-02-14T17:34:53.090 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-02-14T17:34:53.097 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-14T17:34:53.151 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-14T17:34:53.211 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:53.642 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:53.654 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-14T17:34:53.737 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-14T17:34:53.808 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-14T17:34:53.830 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 6/6 2024-02-14T17:34:53.972 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:34:53.972 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:53.972 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-02-14T17:34:53.972 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:34:53.972 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:34:53.972 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-02-14T17:34:54.216 DEBUG:teuthology.parallel:result is None 2024-02-14T17:34:54.675 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 6/6 2024-02-14T17:34:54.676 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/6 2024-02-14T17:34:54.676 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-14T17:34:54.676 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-14T17:34:54.676 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-14T17:34:54.676 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-14T17:34:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-14T17:34:54.984 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:54.985 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:34:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:34:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-14T17:34:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-14T17:34:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-14T17:34:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-14T17:34:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-14T17:34:54.985 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:54.985 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:34:55.264 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-fuse 2024-02-14T17:34:55.845 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:55 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:34:56.010 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:34:56.011 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-14T17:34:56.011 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:56.011 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:34:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 886 k 2024-02-14T17:34:56.011 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-14T17:34:56.011 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-14T17:34:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:56.012 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:34:56.012 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:56.012 INFO:teuthology.orchestra.run.smithi184.stdout:Install 2 Packages 2024-02-14T17:34:56.012 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:56.012 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 965 k 2024-02-14T17:34:56.012 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 2.8 M 2024-02-14T17:34:56.012 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:34:56.159 INFO:teuthology.orchestra.run.smithi184.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 534 kB/s | 79 kB 00:00 2024-02-14T17:34:56.526 INFO:teuthology.orchestra.run.smithi184.stdout:(2/2): ceph-fuse-19.0.0-1430.g8727ce93.el9.x86_ 1.7 MB/s | 886 kB 00:00 2024-02-14T17:34:56.527 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:56.527 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.8 MB/s | 965 kB 00:00 2024-02-14T17:34:56.528 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:34:56.548 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:34:56.548 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:34:56.625 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:34:56.626 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:34:56.807 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:34:56.943 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-14T17:34:57.018 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2/2 2024-02-14T17:34:57.796 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2/2 2024-02-14T17:34:57.797 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/2 2024-02-14T17:34:58.119 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-14T17:34:58.119 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:58.119 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:34:58.119 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-14T17:34:58.120 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:58.120 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:34:58.405 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-volume 2024-02-14T17:34:58.989 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:58 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:34:59.153 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:34:59.154 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:59.154 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-14T17:34:59.154 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:59.154 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:34:59.154 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 264 k 2024-02-14T17:34:59.154 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-14T17:34:59.154 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-14T17:34:59.154 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:59.154 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:34:59.155 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:34:59.155 INFO:teuthology.orchestra.run.smithi184.stdout:Install 2 Packages 2024-02-14T17:34:59.155 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:34:59.155 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 332 k 2024-02-14T17:34:59.155 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 1.5 M 2024-02-14T17:34:59.155 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:34:59.355 INFO:teuthology.orchestra.run.smithi184.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 344 kB/s | 69 kB 00:00 2024-02-14T17:34:59.764 INFO:teuthology.orchestra.run.smithi184.stdout:(2/2): ceph-volume-19.0.0-1430.g8727ce93.el9.no 433 kB/s | 264 kB 00:00 2024-02-14T17:34:59.765 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:34:59.766 INFO:teuthology.orchestra.run.smithi184.stdout:Total 544 kB/s | 332 kB 00:00 2024-02-14T17:34:59.766 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:34:59.774 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:34:59.775 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:34:59.817 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:34:59.817 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:34:59.970 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:35:00.136 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-14T17:35:00.167 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2/2 2024-02-14T17:35:00.182 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2/2 2024-02-14T17:35:00.182 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:35:00.182 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-14T17:35:00.182 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:00.886 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 1/2 2024-02-14T17:35:01.234 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-14T17:35:01.235 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:01.235 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:35:01.235 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:35:01.235 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-14T17:35:01.235 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:01.235 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:01.589 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librados-devel 2024-02-14T17:35:02.172 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:02 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:02.338 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:02.339 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:02.339 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-02-14T17:35:02.339 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:02.339 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:35:02.339 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 127 k 2024-02-14T17:35:02.340 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:02.340 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:35:02.340 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:02.340 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-14T17:35:02.340 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:02.340 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 127 k 2024-02-14T17:35:02.340 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 456 k 2024-02-14T17:35:02.340 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:35:03.428 INFO:teuthology.orchestra.run.smithi184.stdout:librados-devel-19.0.0-1430.g8727ce93.el9.x86_64 116 kB/s | 127 kB 00:01 2024-02-14T17:35:03.428 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:35:03.429 INFO:teuthology.orchestra.run.smithi184.stdout:Total 116 kB/s | 127 kB 00:01 2024-02-14T17:35:03.429 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:35:03.436 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:35:03.436 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:35:03.495 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:35:03.495 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:35:03.703 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:35:03.801 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:04.255 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:04.521 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:04.522 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:35:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:35:04.522 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:04.522 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:04.742 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install libcephfs2 2024-02-14T17:35:05.349 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:05 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:05.470 INFO:teuthology.orchestra.run.smithi184.stdout:Package libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:35:05.522 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:05.523 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-14T17:35:05.523 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:05.579 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install libcephfs-devel 2024-02-14T17:35:06.152 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:06 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:06.316 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:06.317 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:06.317 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-02-14T17:35:06.317 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:06.317 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:35:06.317 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 31 k 2024-02-14T17:35:06.317 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:06.317 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:35:06.317 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:06.317 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-14T17:35:06.317 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:06.318 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 31 k 2024-02-14T17:35:06.318 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 140 k 2024-02-14T17:35:06.318 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:35:06.481 INFO:teuthology.orchestra.run.smithi184.stdout:libcephfs-devel-19.0.0-1430.g8727ce93.el9.x86_6 190 kB/s | 31 kB 00:00 2024-02-14T17:35:06.482 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:35:06.482 INFO:teuthology.orchestra.run.smithi184.stdout:Total 189 kB/s | 31 kB 00:00 2024-02-14T17:35:06.482 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:35:06.486 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:35:06.486 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:35:06.515 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:35:06.515 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:35:06.608 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:35:06.713 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:07.112 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:07.389 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:07.389 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:07.389 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:35:07.390 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:35:07.390 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:07.390 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:07.579 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librados2 2024-02-14T17:35:08.147 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:08 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:08.261 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:35:08.312 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:08.313 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-14T17:35:08.313 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:08.369 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librbd1 2024-02-14T17:35:08.941 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:08 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:09.055 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:35:09.106 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:09.107 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-14T17:35:09.108 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:09.170 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rados 2024-02-14T17:35:09.746 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:09 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:09.859 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:35:09.911 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:09.912 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-14T17:35:09.912 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:09.973 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rgw 2024-02-14T17:35:10.551 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:10 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:10.665 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:35:10.716 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:10.717 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-14T17:35:10.717 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:10.781 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-cephfs 2024-02-14T17:35:11.356 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:11 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:11.468 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:35:11.519 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:11.520 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-14T17:35:11.520 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:11.577 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rbd 2024-02-14T17:35:12.152 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:12 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:12.265 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:35:12.315 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:12.316 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-14T17:35:12.316 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:12.378 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-fuse 2024-02-14T17:35:12.949 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:12 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:13.114 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:13.115 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:13.115 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-02-14T17:35:13.115 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:13.115 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:35:13.115 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 85 k 2024-02-14T17:35:13.115 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:13.115 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:35:13.115 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:13.115 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-14T17:35:13.116 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:13.116 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 85 k 2024-02-14T17:35:13.116 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 231 k 2024-02-14T17:35:13.116 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:35:13.718 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-fuse-19.0.0-1430.g8727ce93.el9.x86_64.rpm 141 kB/s | 85 kB 00:00 2024-02-14T17:35:13.718 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:35:13.718 INFO:teuthology.orchestra.run.smithi184.stdout:Total 141 kB/s | 85 kB 00:00 2024-02-14T17:35:13.718 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:35:13.726 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:35:13.726 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:35:13.782 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:35:13.782 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:35:13.915 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:35:14.025 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:14.398 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:14.703 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:35:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:35:14.703 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:14.703 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:14.952 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-mirror 2024-02-14T17:35:15.543 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:15 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:15.710 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:15.711 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:15.711 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-02-14T17:35:15.711 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:15.711 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:35:15.712 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.1 M 2024-02-14T17:35:15.712 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:15.712 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:35:15.712 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:15.712 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-14T17:35:15.712 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:15.712 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 3.1 M 2024-02-14T17:35:15.712 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 13 M 2024-02-14T17:35:15.712 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:35:16.954 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-mirror-19.0.0-1430.g8727ce93.el9.x86_64.rpm 2.5 MB/s | 3.1 MB 00:01 2024-02-14T17:35:16.954 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:35:16.955 INFO:teuthology.orchestra.run.smithi184.stdout:Total 2.5 MB/s | 3.1 MB 00:01 2024-02-14T17:35:16.955 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:35:16.965 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:35:16.966 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:35:17.042 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:35:17.043 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:35:17.473 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:35:17.521 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:17.545 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:17.545 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:35:17.545 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-14T17:35:17.545 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T17:35:17.545 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T17:35:17.546 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:18.684 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:18.684 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:18.684 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:35:18.684 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:35:18.684 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:18.684 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:18.941 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-nbd 2024-02-14T17:35:19.529 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:19 ago on Wed 14 Feb 2024 05:33:00 PM UTC. 2024-02-14T17:35:19.695 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-14T17:35:19.696 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:19.696 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-02-14T17:35:19.696 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:19.697 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-14T17:35:19.697 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 171 k 2024-02-14T17:35:19.697 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:19.697 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-14T17:35:19.697 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-14T17:35:19.697 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-14T17:35:19.697 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:19.697 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 171 k 2024-02-14T17:35:19.697 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 494 k 2024-02-14T17:35:19.697 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-14T17:35:19.924 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-nbd-19.0.0-1430.g8727ce93.el9.x86_64.rpm 754 kB/s | 171 kB 00:00 2024-02-14T17:35:19.924 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:35:19.924 INFO:teuthology.orchestra.run.smithi184.stdout:Total 749 kB/s | 171 kB 00:00 2024-02-14T17:35:19.925 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-14T17:35:19.932 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-14T17:35:19.932 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-14T17:35:19.990 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-14T17:35:19.991 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-14T17:35:20.147 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-14T17:35:20.255 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:20.681 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:35:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:20.964 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-14T17:35:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:35:20.964 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:35:20.964 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-14T17:35:21.211 DEBUG:teuthology.parallel:result is None 2024-02-14T17:35:21.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:35:21.480 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-14T17:35:21.508 INFO:teuthology.orchestra.run.smithi178.stdout:19.0.0-1430.g8727ce93.el9 2024-02-14T17:35:21.508 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1430.g8727ce93.el9 2024-02-14T17:35:21.509 INFO:teuthology.task.install:The correct ceph version 19.0.0-1430.g8727ce93 is installed. 2024-02-14T17:35:21.510 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:35:21.654 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-14T17:35:21.675 INFO:teuthology.orchestra.run.smithi184.stdout:19.0.0-1430.g8727ce93.el9 2024-02-14T17:35:21.675 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1430.g8727ce93.el9 2024-02-14T17:35:21.676 INFO:teuthology.task.install:The correct ceph version 19.0.0-1430.g8727ce93 is installed. 2024-02-14T17:35:21.677 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-14T17:35:21.677 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:35:21.678 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-14T17:35:21.713 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:35:21.713 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-14T17:35:21.744 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-14T17:35:21.744 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:35:21.744 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-02-14T17:35:21.783 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-14T17:35:21.854 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:35:21.854 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-02-14T17:35:21.882 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-14T17:35:21.949 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-14T17:35:21.949 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:35:21.950 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-14T17:35:21.976 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-14T17:35:22.043 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:35:22.043 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-14T17:35:22.070 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-14T17:35:22.139 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-14T17:35:22.140 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:35:22.140 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-02-14T17:35:22.168 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-14T17:35:22.237 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:35:22.237 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-02-14T17:35:22.264 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-14T17:35:22.331 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-14T17:35:22.429 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'mon_bind_msgr2': False, 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1', 'cephadm_mode': 'cephadm-package'} 2024-02-14T17:35:22.429 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:35:22.430 INFO:tasks.cephadm:Cluster fsid is 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:35:22.430 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-14T17:35:22.430 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.178:6789]', 'mon.c': '[v1:172.21.15.178:6790]', 'mon.b': '[v1:172.21.15.184:6789]'} 2024-02-14T17:35:22.430 INFO:tasks.cephadm:First mon is mon.a on smithi178 2024-02-14T17:35:22.430 INFO:tasks.cephadm:First mgr is y 2024-02-14T17:35:22.431 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-14T17:35:22.431 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2024-02-14T17:35:22.459 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname $(hostname -s) 2024-02-14T17:35:22.501 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 on all hosts... 2024-02-14T17:35:22.502 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 pull 2024-02-14T17:35:22.504 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 pull 2024-02-14T17:35:23.333 INFO:teuthology.orchestra.run.smithi184.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:35:23.359 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:36:10.518 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-02-14T17:36:10.519 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_version": "ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev)", 2024-02-14T17:36:10.519 INFO:teuthology.orchestra.run.smithi184.stdout: "image_id": "170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86", 2024-02-14T17:36:10.519 INFO:teuthology.orchestra.run.smithi184.stdout: "repo_digests": [ 2024-02-14T17:36:10.519 INFO:teuthology.orchestra.run.smithi184.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b" 2024-02-14T17:36:10.519 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-02-14T17:36:10.519 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-02-14T17:36:11.184 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-02-14T17:36:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev)", 2024-02-14T17:36:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86", 2024-02-14T17:36:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2024-02-14T17:36:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b" 2024-02-14T17:36:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-02-14T17:36:11.184 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-02-14T17:36:11.207 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2024-02-14T17:36:11.259 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph 2024-02-14T17:36:11.291 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2024-02-14T17:36:11.318 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /etc/ceph 2024-02-14T17:36:11.358 INFO:tasks.cephadm:Writing seed config... 2024-02-14T17:36:11.359 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-14T17:36:11.359 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-14T17:36:11.359 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-14T17:36:11.359 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-14T17:36:11.359 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-02-14T17:36:11.359 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-14T17:36:11.359 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-14T17:36:11.359 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-14T17:36:11.360 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-14T17:36:11.360 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-02-14T17:36:11.360 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-14T17:36:11.360 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-14T17:36:11.360 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-02-14T17:36:11.360 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-02-14T17:36:11.360 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-14T17:36:11.361 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:36:11.361 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-14T17:36:11.375 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 6e1588ec-cb5f-11ee-95b9-87774f69a715 mon election default strategy = 1 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-02-14T17:36:11.376 DEBUG:teuthology.orchestra.run.smithi178:mon.a> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.a.service 2024-02-14T17:36:11.418 DEBUG:teuthology.orchestra.run.smithi178:mgr.y> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mgr.y.service 2024-02-14T17:36:11.460 INFO:tasks.cephadm:Bootstrapping... 2024-02-14T17:36:11.461 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 -v bootstrap --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.178:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:36:11.652 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:36:11.652 INFO:teuthology.orchestra.run.smithi178.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1', '-v', 'bootstrap', '--fsid', '6e1588ec-cb5f-11ee-95b9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.178:6789]', '--skip-admin-label'] 2024-02-14T17:36:11.677 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T17:36:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-14T17:36:11.677 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying podman|docker is present... 2024-02-14T17:36:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T17:36:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying lvm2 is present... 2024-02-14T17:36:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying time synchronization is in place... 2024-02-14T17:36:11.710 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-14T17:36:11.710 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-14T17:36:11.716 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-14T17:36:11.716 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2024-02-14T17:36:11.722 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout enabled 2024-02-14T17:36:11.728 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout active 2024-02-14T17:36:11.729 INFO:teuthology.orchestra.run.smithi178.stdout:Unit chronyd.service is enabled and running 2024-02-14T17:36:11.729 INFO:teuthology.orchestra.run.smithi178.stdout:Repeating the final host check... 2024-02-14T17:36:11.752 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T17:36:11.752 INFO:teuthology.orchestra.run.smithi178.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-14T17:36:11.752 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl is present 2024-02-14T17:36:11.752 INFO:teuthology.orchestra.run.smithi178.stdout:lvcreate is present 2024-02-14T17:36:11.758 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-14T17:36:11.758 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-14T17:36:11.765 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-14T17:36:11.765 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2024-02-14T17:36:11.770 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout enabled 2024-02-14T17:36:11.776 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout active 2024-02-14T17:36:11.776 INFO:teuthology.orchestra.run.smithi178.stdout:Unit chronyd.service is enabled and running 2024-02-14T17:36:11.776 INFO:teuthology.orchestra.run.smithi178.stdout:Host looks OK 2024-02-14T17:36:11.776 INFO:teuthology.orchestra.run.smithi178.stdout:Cluster fsid: 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:36:11.777 INFO:teuthology.orchestra.run.smithi178.stdout:Acquiring lock 140118474096592 on /run/cephadm/6e1588ec-cb5f-11ee-95b9-87774f69a715.lock 2024-02-14T17:36:11.777 INFO:teuthology.orchestra.run.smithi178.stdout:Lock 140118474096592 acquired on /run/cephadm/6e1588ec-cb5f-11ee-95b9-87774f69a715.lock 2024-02-14T17:36:11.777 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying IP 172.21.15.178 port 6789 ... 2024-02-14T17:36:11.777 INFO:teuthology.orchestra.run.smithi178.stdout:Base mon IP(s) is [172.21.15.178:6789], mon addrv is [v1:172.21.15.178:6789] 2024-02-14T17:36:11.780 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.178 metric 100 2024-02-14T17:36:11.780 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.178 metric 100 2024-02-14T17:36:11.783 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-14T17:36:11.783 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-14T17:36:11.783 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1784sec hoplimit 64 pref medium 2024-02-14T17:36:11.785 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-14T17:36:11.785 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-14T17:36:11.785 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-14T17:36:11.785 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-14T17:36:11.785 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7ef9/64 scope link 2024-02-14T17:36:11.786 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-14T17:36:11.786 INFO:teuthology.orchestra.run.smithi178.stdout:Mon IP `172.21.15.178` is in CIDR network `172.21.0.0/20` 2024-02-14T17:36:11.786 INFO:teuthology.orchestra.run.smithi178.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-02-14T17:36:11.786 INFO:teuthology.orchestra.run.smithi178.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-14T17:36:11.787 INFO:teuthology.orchestra.run.smithi178.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:36:12.288 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stdout 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 2024-02-14T17:36:12.288 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:36:12.288 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Getting image source signatures 2024-02-14T17:36:12.288 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Copying blob sha256:85deddcd78c19bc18770d0f2ed0eee83c11fab2d5e1d2ebb809a77baeec400f6 2024-02-14T17:36:12.288 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-14T17:36:12.288 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Copying config sha256:170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 2024-02-14T17:36:12.289 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-14T17:36:13.166 INFO:teuthology.orchestra.run.smithi178.stdout:ceph: stdout ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev) 2024-02-14T17:36:13.166 INFO:teuthology.orchestra.run.smithi178.stdout:Ceph version: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev) 2024-02-14T17:36:13.167 INFO:teuthology.orchestra.run.smithi178.stdout:Extracting ceph user uid/gid from container image... 2024-02-14T17:36:14.191 INFO:teuthology.orchestra.run.smithi178.stdout:stat: stdout 167 167 2024-02-14T17:36:14.191 INFO:teuthology.orchestra.run.smithi178.stdout:Creating initial keys... 2024-02-14T17:36:15.097 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph-authtool: stdout AQAO+sxljDVZIBAA3825mnRg+E3FvTFciNjHhA== 2024-02-14T17:36:16.044 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph-authtool: stdout AQAP+sxlcllKGhAACDjGDDiKr4/tBYc/0t5wHQ== 2024-02-14T17:36:16.992 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph-authtool: stdout AQAQ+sxlwTMfGRAAz4y3imyAlcrA/CerpI+tkw== 2024-02-14T17:36:16.996 INFO:teuthology.orchestra.run.smithi178.stdout:Creating initial monmap... 2024-02-14T17:36:17.942 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-14T17:36:17.942 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-14T17:36:17.942 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:36:17.942 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-14T17:36:17.942 INFO:teuthology.orchestra.run.smithi178.stdout:monmaptool for a [v1:172.21.15.178:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-14T17:36:17.942 INFO:teuthology.orchestra.run.smithi178.stdout:setting min_mon_release = quincy 2024-02-14T17:36:17.942 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: set fsid to 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:36:17.942 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-14T17:36:17.942 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:36:17.942 INFO:teuthology.orchestra.run.smithi178.stdout:Creating mon... 2024-02-14T17:36:18.931 INFO:teuthology.orchestra.run.smithi178.stdout:create mon.a on 2024-02-14T17:36:19.128 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-14T17:36:19.258 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-14T17:36:19.422 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715.target. 2024-02-14T17:36:19.422 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715.target. 2024-02-14T17:36:19.682 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.a 2024-02-14T17:36:19.683 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.a.service: Unit ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.a.service not loaded. 2024-02-14T17:36:19.839 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715.target.wants/ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@.service. 2024-02-14T17:36:20.396 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:36:20.397 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout disabled 2024-02-14T17:36:20.403 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:36:20.403 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2024-02-14T17:36:20.403 INFO:teuthology.orchestra.run.smithi178.stdout:firewalld.service is not enabled 2024-02-14T17:36:20.403 INFO:teuthology.orchestra.run.smithi178.stdout:Not possible to enable service . firewalld.service is not available 2024-02-14T17:36:20.404 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mon to start... 2024-02-14T17:36:20.404 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mon... 2024-02-14T17:36:21.524 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout cluster: 2024-02-14T17:36:21.524 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout id: 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:36:21.524 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-14T17:36:21.524 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout services: 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.596782s) 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout data: 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout pgs: 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.851+0000 7f265b0ee700 1 Processor -- start 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.851+0000 7f265b0ee700 1 -- start start 2024-02-14T17:36:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.852+0000 7f265b0ee700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2654109f10 con 0x7f2654105ae0 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.852+0000 7f2658e8a700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2654105ae0 0x7f2654105f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43386/0 (socket says 172.21.15.178:43386) 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.852+0000 7f2658e8a700 1 -- 172.21.15.178:0/2669130306 learned_addr learned my addr 172.21.15.178:0/2669130306 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.852+0000 7f26537fe700 1 -- 172.21.15.178:0/2669130306 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305077175 0 0) 0x7f2654109f10 con 0x7f2654105ae0 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.852+0000 7f26537fe700 1 -- 172.21.15.178:0/2669130306 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26440035b0 con 0x7f2654105ae0 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.852+0000 7f26537fe700 1 -- 172.21.15.178:0/2669130306 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1082493448 0 0) 0x7f26440035b0 con 0x7f2654105ae0 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.852+0000 7f26537fe700 1 -- 172.21.15.178:0/2669130306 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265410b0b0 con 0x7f2654105ae0 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.852+0000 7f26537fe700 1 -- 172.21.15.178:0/2669130306 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2648001580 con 0x7f2654105ae0 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.853+0000 7f26537fe700 1 -- 172.21.15.178:0/2669130306 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2648001870 con 0x7f2654105ae0 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.853+0000 7f26537fe700 1 -- 172.21.15.178:0/2669130306 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2648001bb0 con 0x7f2654105ae0 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.853+0000 7f265b0ee700 1 -- 172.21.15.178:0/2669130306 >> v1:172.21.15.178:6789/0 conn(0x7f2654105ae0 legacy=0x7f2654105f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.853+0000 7f265b0ee700 1 -- 172.21.15.178:0/2669130306 shutdown_connections 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.853+0000 7f265b0ee700 1 -- 172.21.15.178:0/2669130306 >> 172.21.15.178:0/2669130306 conn(0x7f2654101040 msgr2=0x7f26541034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:21.526 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.853+0000 7f265b0ee700 1 -- 172.21.15.178:0/2669130306 shutdown_connections 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.853+0000 7f265b0ee700 1 -- 172.21.15.178:0/2669130306 wait complete. 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.854+0000 7f265b0ee700 1 Processor -- start 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.854+0000 7f265b0ee700 1 -- start start 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.854+0000 7f265b0ee700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f26541992e0 con 0x7f2654105ae0 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.854+0000 7f2658e8a700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2654105ae0 0x7f2654198bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43398/0 (socket says 172.21.15.178:43398) 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.854+0000 7f2658e8a700 1 -- 172.21.15.178:0/199813711 learned_addr learned my addr 172.21.15.178:0/199813711 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.854+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 535546701 0 0) 0x7f26541992e0 con 0x7f2654105ae0 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.854+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26440031f0 con 0x7f2654105ae0 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.854+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 896528230 0 0) 0x7f26440031f0 con 0x7f2654105ae0 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.854+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f26541992e0 con 0x7f2654105ae0 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.854+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2648001540 con 0x7f2654105ae0 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.855+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 728205756 0 0) 0x7f26541992e0 con 0x7f2654105ae0 2024-02-14T17:36:21.527 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.855+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2654199470 con 0x7f2654105ae0 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.855+0000 7f265b0ee700 1 -- 172.21.15.178:0/199813711 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2654199720 con 0x7f2654105ae0 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.855+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2648004b90 con 0x7f2654105ae0 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.855+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2648004e90 con 0x7f2654105ae0 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.855+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7f26480053c0 con 0x7f2654105ae0 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.855+0000 7f265b0ee700 1 -- 172.21.15.178:0/199813711 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f265419d270 con 0x7f2654105ae0 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.855+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 1943377556 0 0) 0x7f2648005be0 con 0x7f2654105ae0 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.856+0000 7f265b0ee700 1 -- 172.21.15.178:0/199813711 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2654192e70 con 0x7f2654105ae0 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.858+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f2648006790 con 0x7f2654105ae0 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.928+0000 7f265b0ee700 1 -- 172.21.15.178:0/199813711 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f265410a1e0 con 0x7f2654105ae0 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.929+0000 7f2651ffb700 1 -- 172.21.15.178:0/199813711 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (unknown 1155462804 0 173586983) 0x7f26480069f0 con 0x7f2654105ae0 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.930+0000 7f265b0ee700 1 -- 172.21.15.178:0/199813711 >> v1:172.21.15.178:6789/0 conn(0x7f2654105ae0 legacy=0x7f2654198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.930+0000 7f265b0ee700 1 -- 172.21.15.178:0/199813711 shutdown_connections 2024-02-14T17:36:21.528 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.930+0000 7f265b0ee700 1 -- 172.21.15.178:0/199813711 >> 172.21.15.178:0/199813711 conn(0x7f2654101040 msgr2=0x7f265418ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:21.529 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.930+0000 7f265b0ee700 1 -- 172.21.15.178:0/199813711 shutdown_connections 2024-02-14T17:36:21.529 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:20.930+0000 7f265b0ee700 1 -- 172.21.15.178:0/199813711 wait complete. 2024-02-14T17:36:21.529 INFO:teuthology.orchestra.run.smithi178.stdout:mon is available 2024-02-14T17:36:21.529 INFO:teuthology.orchestra.run.smithi178.stdout:Assimilating anything we can from ceph.conf... 2024-02-14T17:36:22.590 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:22.590 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [global] 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout fsid = 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.178:6789] 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [osd] 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.959+0000 7f927e71b700 1 Processor -- start 2024-02-14T17:36:22.591 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.959+0000 7f927e71b700 1 -- start start 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.960+0000 7f927e71b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9278108e30 con 0x7f9278104a00 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.960+0000 7f9277fff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f9278104a00 0x7f9278104e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43410/0 (socket says 172.21.15.178:43410) 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.960+0000 7f9277fff700 1 -- 172.21.15.178:0/2141698634 learned_addr learned my addr 172.21.15.178:0/2141698634 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.960+0000 7f9276ffd700 1 -- 172.21.15.178:0/2141698634 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081904016 0 0) 0x7f9278108e30 con 0x7f9278104a00 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.960+0000 7f9276ffd700 1 -- 172.21.15.178:0/2141698634 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92640035b0 con 0x7f9278104a00 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.960+0000 7f9276ffd700 1 -- 172.21.15.178:0/2141698634 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2477277426 0 0) 0x7f92640035b0 con 0x7f9278104a00 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.960+0000 7f9276ffd700 1 -- 172.21.15.178:0/2141698634 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9278109fd0 con 0x7f9278104a00 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.960+0000 7f9276ffd700 1 -- 172.21.15.178:0/2141698634 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f9268001580 con 0x7f9278104a00 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.960+0000 7f9276ffd700 1 -- 172.21.15.178:0/2141698634 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9268001870 con 0x7f9278104a00 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.960+0000 7f9276ffd700 1 -- 172.21.15.178:0/2141698634 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f9268001bb0 con 0x7f9278104a00 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.961+0000 7f927e71b700 1 -- 172.21.15.178:0/2141698634 >> v1:172.21.15.178:6789/0 conn(0x7f9278104a00 legacy=0x7f9278104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.961+0000 7f927e71b700 1 -- 172.21.15.178:0/2141698634 shutdown_connections 2024-02-14T17:36:22.592 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.961+0000 7f927e71b700 1 -- 172.21.15.178:0/2141698634 >> 172.21.15.178:0/2141698634 conn(0x7f92780fff80 msgr2=0x7f92781023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.961+0000 7f927e71b700 1 -- 172.21.15.178:0/2141698634 shutdown_connections 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.961+0000 7f927e71b700 1 -- 172.21.15.178:0/2141698634 wait complete. 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.961+0000 7f927e71b700 1 Processor -- start 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.962+0000 7f927e71b700 1 -- start start 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.962+0000 7f927e71b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f92781992f0 con 0x7f9278104a00 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.962+0000 7f9277fff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f9278104a00 0x7f9278198be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43418/0 (socket says 172.21.15.178:43418) 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.962+0000 7f9277fff700 1 -- 172.21.15.178:0/1787091488 learned_addr learned my addr 172.21.15.178:0/1787091488 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.962+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317193163 0 0) 0x7f92781992f0 con 0x7f9278104a00 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.962+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92640031f0 con 0x7f9278104a00 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.962+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2310788195 0 0) 0x7f92640031f0 con 0x7f9278104a00 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.962+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f92781992f0 con 0x7f9278104a00 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.962+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f9268001540 con 0x7f9278104a00 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.963+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3837625478 0 0) 0x7f92781992f0 con 0x7f9278104a00 2024-02-14T17:36:22.593 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.963+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9278199480 con 0x7f9278104a00 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.963+0000 7f927e71b700 1 -- 172.21.15.178:0/1787091488 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9278199730 con 0x7f9278104a00 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.963+0000 7f927e71b700 1 -- 172.21.15.178:0/1787091488 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f927819d280 con 0x7f9278104a00 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.963+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9268004b20 con 0x7f9278104a00 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.963+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f9268004e20 con 0x7f9278104a00 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.963+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7f92680053d0 con 0x7f9278104a00 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.963+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 1943377556 0 0) 0x7f9268006650 con 0x7f9278104a00 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.964+0000 7f927e71b700 1 -- 172.21.15.178:0/1787091488 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9258005150 con 0x7f9278104a00 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:21.966+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f9268005810 con 0x7f9278104a00 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:22.033+0000 7f927e71b700 1 -- 172.21.15.178:0/1787091488 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f9258005980 con 0x7f9278104a00 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:22.035+0000 7f92757fa700 1 -- 172.21.15.178:0/1787091488 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+318 (unknown 1213389831 0 1434013398) 0x7f9268005a70 con 0x7f9278104a00 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:22.036+0000 7f927e71b700 1 -- 172.21.15.178:0/1787091488 >> v1:172.21.15.178:6789/0 conn(0x7f9278104a00 legacy=0x7f9278198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:22.036+0000 7f927e71b700 1 -- 172.21.15.178:0/1787091488 shutdown_connections 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:22.036+0000 7f927e71b700 1 -- 172.21.15.178:0/1787091488 >> 172.21.15.178:0/1787091488 conn(0x7f92780fff80 msgr2=0x7f92780783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:22.036+0000 7f927e71b700 1 -- 172.21.15.178:0/1787091488 shutdown_connections 2024-02-14T17:36:22.594 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:22.036+0000 7f927e71b700 1 -- 172.21.15.178:0/1787091488 wait complete. 2024-02-14T17:36:22.595 INFO:teuthology.orchestra.run.smithi178.stdout:Generating new minimal ceph.conf... 2024-02-14T17:36:23.429 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:23 smithi178 ceph-mon[29831]: from='client.? v1:172.21.15.178:0/3305891111' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:36:25.984 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.049+0000 7f1661f18700 1 Processor -- start 2024-02-14T17:36:25.984 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.049+0000 7f1661f18700 1 -- start start 2024-02-14T17:36:25.984 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.049+0000 7f1661f18700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f165c078df0 con 0x7f165c106650 2024-02-14T17:36:25.984 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.049+0000 7f165b7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f165c106650 0x7f165c108a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43428/0 (socket says 172.21.15.178:43428) 2024-02-14T17:36:25.984 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.049+0000 7f165b7fe700 1 -- 172.21.15.178:0/1901577417 learned_addr learned my addr 172.21.15.178:0/1901577417 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:25.984 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.050+0000 7f165a7fc700 1 -- 172.21.15.178:0/1901577417 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674294258 0 0) 0x7f165c078df0 con 0x7f165c106650 2024-02-14T17:36:25.984 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.050+0000 7f165a7fc700 1 -- 172.21.15.178:0/1901577417 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16440035b0 con 0x7f165c106650 2024-02-14T17:36:25.984 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.050+0000 7f165a7fc700 1 -- 172.21.15.178:0/1901577417 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3380791601 0 0) 0x7f16440035b0 con 0x7f165c106650 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.050+0000 7f165a7fc700 1 -- 172.21.15.178:0/1901577417 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c10b170 con 0x7f165c106650 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.050+0000 7f165a7fc700 1 -- 172.21.15.178:0/1901577417 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f1650001580 con 0x7f165c106650 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.050+0000 7f165a7fc700 1 -- 172.21.15.178:0/1901577417 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f1650001c10 con 0x7f165c106650 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.050+0000 7f165a7fc700 1 -- 172.21.15.178:0/1901577417 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f16500057b0 con 0x7f165c106650 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.051+0000 7f1661f18700 1 -- 172.21.15.178:0/1901577417 >> v1:172.21.15.178:6789/0 conn(0x7f165c106650 legacy=0x7f165c108a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.051+0000 7f1661f18700 1 -- 172.21.15.178:0/1901577417 shutdown_connections 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.051+0000 7f1661f18700 1 -- 172.21.15.178:0/1901577417 >> 172.21.15.178:0/1901577417 conn(0x7f165c0fff80 msgr2=0x7f165c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.051+0000 7f1661f18700 1 -- 172.21.15.178:0/1901577417 shutdown_connections 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.051+0000 7f1661f18700 1 -- 172.21.15.178:0/1901577417 wait complete. 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.051+0000 7f1661f18700 1 Processor -- start 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f1661f18700 1 -- start start 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f1661f18700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f165c199b80 con 0x7f165c106650 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f165b7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f165c106650 0x7f165c199470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43444/0 (socket says 172.21.15.178:43444) 2024-02-14T17:36:25.985 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f165b7fe700 1 -- 172.21.15.178:0/3305891111 learned_addr learned my addr 172.21.15.178:0/3305891111 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199118052 0 0) 0x7f165c199b80 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1644005540 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3984983832 0 0) 0x7f1644005540 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f165c199b80 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f1650001080 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3215998764 0 0) 0x7f165c199b80 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c199d10 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.052+0000 7f1661f18700 1 -- 172.21.15.178:0/3305891111 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165c199fc0 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.053+0000 7f1661f18700 1 -- 172.21.15.178:0/3305891111 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f165c19dac0 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.053+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f16500049d0 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.053+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f16500062c0 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.053+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7f16500068b0 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.053+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 1943377556 0 0) 0x7f1650007da0 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.053+0000 7f1661f18700 1 -- 172.21.15.178:0/3305891111 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1648005120 con 0x7f165c106650 2024-02-14T17:36:25.986 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.056+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f1650005b20 con 0x7f165c106650 2024-02-14T17:36:25.987 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.124+0000 7f1661f18700 1 -- 172.21.15.178:0/3305891111 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f16480053d0 con 0x7f165c106650 2024-02-14T17:36:25.987 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.125+0000 7f1658ff9700 1 -- 172.21.15.178:0/3305891111 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+148 (unknown 1452402520 0 192482225) 0x7f1650004e70 con 0x7f165c106650 2024-02-14T17:36:25.987 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.126+0000 7f1661f18700 1 -- 172.21.15.178:0/3305891111 >> v1:172.21.15.178:6789/0 conn(0x7f165c106650 legacy=0x7f165c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:25.987 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.126+0000 7f1661f18700 1 -- 172.21.15.178:0/3305891111 shutdown_connections 2024-02-14T17:36:25.987 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.126+0000 7f1661f18700 1 -- 172.21.15.178:0/3305891111 >> 172.21.15.178:0/3305891111 conn(0x7f165c0fff80 msgr2=0x7f165c190270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:25.987 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.126+0000 7f1661f18700 1 -- 172.21.15.178:0/3305891111 shutdown_connections 2024-02-14T17:36:25.987 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:23.126+0000 7f1661f18700 1 -- 172.21.15.178:0/3305891111 wait complete. 2024-02-14T17:36:25.987 INFO:teuthology.orchestra.run.smithi178.stdout:Restarting the monitor... 2024-02-14T17:36:26.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:26 smithi178 systemd[1]: Stopping Ceph mon.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:36:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:27 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a[29808]: 2024-02-14T17:36:27.018+0000 7f05cdc4d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T17:36:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:27 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a[29808]: 2024-02-14T17:36:27.018+0000 7f05cdc4d700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-14T17:36:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:27 smithi178 podman[30163]: 2024-02-14 17:36:27.162175512 +0000 UTC m=+1.008525432 container died da130b3b4d16392067951a8936221b97ef0ef2e22707d1c9f5080cae4a683c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux ) 2024-02-14T17:36:31.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:31 smithi178 podman[30163]: 2024-02-14 17:36:31.047103099 +0000 UTC m=+4.893453006 container cleanup da130b3b4d16392067951a8936221b97ef0ef2e22707d1c9f5080cae4a683c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-14T17:36:31.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:31 smithi178 bash[30163]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a 2024-02-14T17:36:32.753 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:32 smithi178 podman[30182]: 2024-02-14 17:36:32.443769031 +0000 UTC m=+5.275126199 container remove da130b3b4d16392067951a8936221b97ef0ef2e22707d1c9f5080cae4a683c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-14T17:36:32.754 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:32 smithi178 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-14T17:36:32.754 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:32 smithi178 systemd[1]: Stopped Ceph mon.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:36:32.754 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:32 smithi178 systemd[1]: Starting Ceph mon.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:36:33.048 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:32 smithi178 podman[30282]: 2024-02-14 17:36:32.733497728 +0000 UTC m=+0.019132970 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:36:33.048 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:32 smithi178 podman[30282]: 2024-02-14 17:36:32.853198454 +0000 UTC m=+0.138833695 container create 46ad1d3e6f4404dd1ea1a53a400fcb84fa8160fbe44b330b5216e28512550b2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-14T17:36:33.048 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 podman[30282]: 2024-02-14 17:36:33.047141891 +0000 UTC m=+0.332777130 container init 46ad1d3e6f4404dd1ea1a53a400fcb84fa8160fbe44b330b5216e28512550b2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T17:36:33.127 INFO:teuthology.orchestra.run.smithi178.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-14T17:36:33.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 podman[30282]: 2024-02-14 17:36:33.052776216 +0000 UTC m=+0.338411455 container start 46ad1d3e6f4404dd1ea1a53a400fcb84fa8160fbe44b330b5216e28512550b2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T17:36:33.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: set uid:gid to 167:167 (ceph:ceph) 2024-02-14T17:36:33.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev), process ceph-mon, pid 8 2024-02-14T17:36:33.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: pidfile_write: ignore empty --pid-file 2024-02-14T17:36:33.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: load: jerasure load: lrc 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: RocksDB version: 7.9.2 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Git sha 0 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Compile date 2024-02-14 11:59:27 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: DB SUMMARY 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: DB Session ID: J8NTYMSPQ7GQK3F9NOTH 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: CURRENT file: CURRENT 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: IDENTITY file: IDENTITY 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 85566 ; 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.error_if_exists: 0 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.create_if_missing: 0 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.paranoid_checks: 1 2024-02-14T17:36:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.env: 0x556d7ed39ae0 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.fs: PosixFileSystem 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.info_log: 0x556d81435900 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_file_opening_threads: 16 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.statistics: (nil) 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.use_fsync: 0 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_log_file_size: 0 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.keep_log_file_num: 1000 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.recycle_log_file_num: 0 2024-02-14T17:36:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.allow_fallocate: 1 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.allow_mmap_reads: 0 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.allow_mmap_writes: 0 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.use_direct_reads: 0 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.create_missing_column_families: 0 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.db_log_dir: 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.wal_dir: 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.advise_random_on_open: 1 2024-02-14T17:36:33.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.db_write_buffer_size: 0 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.write_buffer_manager: 0x556d81409400 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.rate_limiter: (nil) 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.wal_recovery_mode: 2 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.enable_thread_tracking: 0 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.enable_pipelined_write: 0 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.unordered_write: 0 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-14T17:36:33.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-14T17:36:33.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.row_cache: None 2024-02-14T17:36:33.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.wal_filter: None 2024-02-14T17:36:33.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-14T17:36:33.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.allow_ingest_behind: 0 2024-02-14T17:36:33.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.two_write_queues: 0 2024-02-14T17:36:33.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.manual_wal_flush: 0 2024-02-14T17:36:33.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.wal_compression: 0 2024-02-14T17:36:33.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.atomic_flush: 0 2024-02-14T17:36:33.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-14T17:36:33.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-14T17:36:33.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-14T17:36:33.339 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.log_readahead_size: 0 2024-02-14T17:36:33.339 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-14T17:36:33.339 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.best_efforts_recovery: 0 2024-02-14T17:36:33.339 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-14T17:36:33.339 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-14T17:36:33.339 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.allow_data_in_errors: 0 2024-02-14T17:36:33.339 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.db_host_id: __hostname__ 2024-02-14T17:36:33.339 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-14T17:36:33.339 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_background_jobs: 2 2024-02-14T17:36:33.339 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_background_compactions: -1 2024-02-14T17:36:33.339 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_subcompactions: 1 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_total_wal_size: 0 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_open_files: -1 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bytes_per_sync: 0 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_readahead_size: 0 2024-02-14T17:36:33.340 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_background_flushes: -1 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Compression algorithms supported: 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: kZSTD supported: 0 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: kXpressCompression supported: 0 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: kLZ4HCCompression supported: 1 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: kZlibCompression supported: 1 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: kSnappyCompression supported: 1 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: kLZ4Compression supported: 1 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: kBZip2Compression supported: 0 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-14T17:36:33.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.merge_operator: 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_filter: None 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_filter_factory: None 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.sst_partitioner_factory: None 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556d81435a00) 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout: cache_index_and_filter_blocks: 1 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout: pin_top_level_index_and_filter: 1 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout: index_type: 0 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout: data_block_index_type: 0 2024-02-14T17:36:33.342 INFO:journalctl@ceph.mon.a.smithi178.stdout: index_shortening: 1 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: checksum: 4 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: no_block_cache: 0 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_cache: 0x556d81439090 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_cache_name: BinnedLRUCache 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_cache_options: 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: capacity : 536870912 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: num_shard_bits : 4 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: strict_capacity_limit : 0 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: high_pri_pool_ratio: 0.000 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_cache_compressed: (nil) 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: persistent_cache: (nil) 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_size: 4096 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_size_deviation: 10 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_restart_interval: 16 2024-02-14T17:36:33.343 INFO:journalctl@ceph.mon.a.smithi178.stdout: index_block_restart_interval: 1 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: metadata_block_size: 4096 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: partition_filters: 0 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: use_delta_encoding: 1 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: filter_policy: bloomfilter 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: whole_key_filtering: 1 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: verify_compression: 0 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: read_amp_bytes_per_bit: 0 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: format_version: 5 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: enable_index_compression: 1 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_align: 0 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: max_auto_readahead_size: 262144 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: prepopulate_block_cache: 0 2024-02-14T17:36:33.344 INFO:journalctl@ceph.mon.a.smithi178.stdout: initial_auto_readahead_size: 8192 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.write_buffer_size: 33554432 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_write_buffer_number: 2 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compression: NoCompression 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bottommost_compression: Disabled 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.prefix_extractor: nullptr 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.num_levels: 7 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-14T17:36:33.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compression_opts.level: 32767 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compression_opts.strategy: 0 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compression_opts.enabled: false 2024-02-14T17:36:33.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.target_file_size_base: 67108864 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-14T17:36:33.347 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.arena_block_size: 1048576 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.disable_auto_compactions: 0 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-14T17:36:33.348 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.table_properties_collectors: 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.inplace_update_support: 0 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.bloom_locality: 0 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.max_successive_merges: 0 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.paranoid_file_checks: 0 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.force_consistency_checks: 1 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.report_bg_io_stats: 0 2024-02-14T17:36:33.349 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.ttl: 2592000 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.enable_blob_files: false 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.min_blob_size: 0 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.blob_file_size: 268435456 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.blob_file_starting_level: 0 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-14T17:36:33.350 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0e3ca6ca-7f49-46a7-a4b7-c3b82f6addac 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932193081658, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932193082516, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81900, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 264, "table_properties": {"data_size": 80099, "index_size": 211, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10703, "raw_average_key_size": 47, "raw_value_size": 74074, "raw_average_value_size": 332, "num_data_blocks": 10, "num_entries": 223, "num_filter_entries": 223, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707932193, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0e3ca6ca-7f49-46a7-a4b7-c3b82f6addac", "db_session_id": "J8NTYMSPQ7GQK3F9NOTH", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932193082595, "job": 1, "event": "recovery_finished"} 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556d8152a000 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: DB pointer 0x556d81514000 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout: ** DB Stats ** 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:36:33.351 INFO:journalctl@ceph.mon.a.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: ** Compaction Stats [default] ** 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: L0 2/0 81.78 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: Sum 2/0 81.78 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2024-02-14T17:36:33.352 INFO:journalctl@ceph.mon.a.smithi178.stdout: ** Compaction Stats [default] ** 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 142.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: Cumulative compaction: 0.00 GB write, 27.74 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:36:33.353 INFO:journalctl@ceph.mon.a.smithi178.stdout: Interval compaction: 0.00 GB write, 27.74 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout: Block cache BinnedLRUCache@0x556d81439090#8 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: starting mon.a rank 0 at public addrs v1:172.21.15.178:6789/0 at bind addrs v1:172.21.15.178:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: mon.a@-1(???) e1 preinit fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: mon.a@-1(???).mds e1 new map 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: mon.a@-1(???).mds e1 print_map 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout: e1 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout: legacy client fscid: -1 2024-02-14T17:36:33.354 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout: No filesystems configured 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 bash[30282]: 46ad1d3e6f4404dd1ea1a53a400fcb84fa8160fbe44b330b5216e28512550b2b 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 systemd[1]: Started Ceph mon.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: monmap e1: 1 mons at {a=v1:172.21.15.178:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: fsmap 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: osdmap e1: 0 total, 0 up, 0 in 2024-02-14T17:36:33.355 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:33 smithi178 ceph-mon[30321]: mgrmap e1: no daemons active 2024-02-14T17:36:34.220 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.562+0000 7f5940e26700 1 Processor -- start 2024-02-14T17:36:34.220 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.563+0000 7f5940e26700 1 -- start start 2024-02-14T17:36:34.220 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.563+0000 7f5940e26700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f593c109f40 con 0x7f593c105b10 2024-02-14T17:36:34.220 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.563+0000 7f593a59c700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f593c105b10 0x7f593c105f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43142/0 (socket says 172.21.15.178:43142) 2024-02-14T17:36:34.220 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.563+0000 7f593a59c700 1 -- 172.21.15.178:0/47584996 learned_addr learned my addr 172.21.15.178:0/47584996 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:34.220 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.564+0000 7f593959a700 1 -- 172.21.15.178:0/47584996 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614500769 0 0) 0x7f593c109f40 con 0x7f593c105b10 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.564+0000 7f593959a700 1 -- 172.21.15.178:0/47584996 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59240035b0 con 0x7f593c105b10 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.564+0000 7f593959a700 1 -- 172.21.15.178:0/47584996 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2894293585 0 0) 0x7f59240035b0 con 0x7f593c105b10 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.564+0000 7f593959a700 1 -- 172.21.15.178:0/47584996 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c10b0e0 con 0x7f593c105b10 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.564+0000 7f593959a700 1 -- 172.21.15.178:0/47584996 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f5930001580 con 0x7f593c105b10 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.564+0000 7f593959a700 1 -- 172.21.15.178:0/47584996 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f5930001ce0 con 0x7f593c105b10 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.564+0000 7f593959a700 1 -- 172.21.15.178:0/47584996 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f5930005960 con 0x7f593c105b10 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.564+0000 7f5940e26700 1 -- 172.21.15.178:0/47584996 >> v1:172.21.15.178:6789/0 conn(0x7f593c105b10 legacy=0x7f593c105f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.565+0000 7f5940e26700 1 -- 172.21.15.178:0/47584996 shutdown_connections 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.565+0000 7f5940e26700 1 -- 172.21.15.178:0/47584996 >> 172.21.15.178:0/47584996 conn(0x7f593c1010b0 msgr2=0x7f593c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.565+0000 7f5940e26700 1 -- 172.21.15.178:0/47584996 shutdown_connections 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.565+0000 7f5940e26700 1 -- 172.21.15.178:0/47584996 wait complete. 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.565+0000 7f5940e26700 1 Processor -- start 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.565+0000 7f5940e26700 1 -- start start 2024-02-14T17:36:34.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f5940e26700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f593c199b20 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f593a59c700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f593c105b10 0x7f593c199410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43156/0 (socket says 172.21.15.178:43156) 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f593a59c700 1 -- 172.21.15.178:0/1552670895 learned_addr learned my addr 172.21.15.178:0/1552670895 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58919988 0 0) 0x7f593c199b20 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59240054f0 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1390959144 0 0) 0x7f59240054f0 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f593c199b20 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f5930005940 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 709955054 0 0) 0x7f593c199b20 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c199cb0 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f5940e26700 1 -- 172.21.15.178:0/1552670895 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593c199f60 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f5940e26700 1 -- 172.21.15.178:0/1552670895 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f593c19dab0 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f5930001b20 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f59300061e0 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7f59300067f0 con 0x7f593c105b10 2024-02-14T17:36:34.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.566+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 1943377556 0 0) 0x7f5930007db0 con 0x7f593c105b10 2024-02-14T17:36:34.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.567+0000 7f5940e26700 1 -- 172.21.15.178:0/1552670895 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c005150 con 0x7f593c105b10 2024-02-14T17:36:34.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.569+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f5930005690 con 0x7f593c105b10 2024-02-14T17:36:34.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.637+0000 7f5940e26700 1 -- 172.21.15.178:0/1552670895 --> v1:172.21.15.178:6789/0 -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f591c005980 con 0x7f593c105b10 2024-02-14T17:36:34.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.639+0000 7f592b7fe700 1 -- 172.21.15.178:0/1552670895 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (unknown 534127980 0 0) 0x7f5930005c40 con 0x7f593c105b10 2024-02-14T17:36:34.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.640+0000 7f5940e26700 1 -- 172.21.15.178:0/1552670895 >> v1:172.21.15.178:6789/0 conn(0x7f593c105b10 legacy=0x7f593c199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:34.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.640+0000 7f5940e26700 1 -- 172.21.15.178:0/1552670895 shutdown_connections 2024-02-14T17:36:34.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.640+0000 7f5940e26700 1 -- 172.21.15.178:0/1552670895 >> 172.21.15.178:0/1552670895 conn(0x7f593c1010b0 msgr2=0x7f593c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:34.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.640+0000 7f5940e26700 1 -- 172.21.15.178:0/1552670895 shutdown_connections 2024-02-14T17:36:34.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:33.640+0000 7f5940e26700 1 -- 172.21.15.178:0/1552670895 wait complete. 2024-02-14T17:36:34.223 INFO:teuthology.orchestra.run.smithi178.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-14T17:36:34.282 INFO:teuthology.orchestra.run.smithi178.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:36:34.282 INFO:teuthology.orchestra.run.smithi178.stdout:Creating mgr... 2024-02-14T17:36:34.282 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-14T17:36:34.283 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-14T17:36:34.523 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mgr.y 2024-02-14T17:36:34.523 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mgr.y.service: Unit ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mgr.y.service not loaded. 2024-02-14T17:36:34.664 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715.target.wants/ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@.service. 2024-02-14T17:36:34.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1552670895' entity='client.admin' 2024-02-14T17:36:34.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:34 smithi178 systemd[1]: Starting Ceph mgr.y for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:36:35.210 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:34 smithi178 podman[30564]: 2024-02-14 17:36:34.799777588 +0000 UTC m=+0.017999697 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:36:35.210 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:34 smithi178 podman[30564]: 2024-02-14 17:36:34.897837098 +0000 UTC m=+0.116059205 container create 8e89fc86ca955baa915fb52ae9c8412eb23942aee79c419a6ca33369ba717773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-02-14T17:36:35.210 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:35 smithi178 podman[30564]: 2024-02-14 17:36:35.100417972 +0000 UTC m=+0.318640078 container init 8e89fc86ca955baa915fb52ae9c8412eb23942aee79c419a6ca33369ba717773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-14T17:36:35.210 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:35 smithi178 podman[30564]: 2024-02-14 17:36:35.105604796 +0000 UTC m=+0.323826899 container start 8e89fc86ca955baa915fb52ae9c8412eb23942aee79c419a6ca33369ba717773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-14T17:36:35.210 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:35 smithi178 bash[30564]: 8e89fc86ca955baa915fb52ae9c8412eb23942aee79c419a6ca33369ba717773 2024-02-14T17:36:35.223 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:36:35.223 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout disabled 2024-02-14T17:36:35.230 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:36:35.230 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2024-02-14T17:36:35.230 INFO:teuthology.orchestra.run.smithi178.stdout:firewalld.service is not enabled 2024-02-14T17:36:35.231 INFO:teuthology.orchestra.run.smithi178.stdout:Not possible to enable service . firewalld.service is not available 2024-02-14T17:36:35.239 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:36:35.239 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout disabled 2024-02-14T17:36:35.246 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:36:35.246 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2024-02-14T17:36:35.246 INFO:teuthology.orchestra.run.smithi178.stdout:firewalld.service is not enabled 2024-02-14T17:36:35.246 INFO:teuthology.orchestra.run.smithi178.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-14T17:36:35.246 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mgr to start... 2024-02-14T17:36:35.246 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mgr... 2024-02-14T17:36:35.500 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:35 smithi178 systemd[1]: Started Ceph mgr.y for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:36:35.768 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:35.675+0000 7fb53e5cc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:36:36.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1590864658' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T17:36:36.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:35.774+0000 7fb53e5cc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:36:36.447 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:36.447 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:36:36.447 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "fsid": "6e1588ec-cb5f-11ee-95b9-87774f69a715", 2024-02-14T17:36:36.447 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 0 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-14T17:36:36.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T17:36:36.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:36.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T17:36:36.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T17:36:36.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:36.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T17:36:36.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:36.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T17:36:36.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T17:36:36.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T17:36:36.458 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T17:36:36.458 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T17:36:36.458 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T17:36:36.458 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:36.458 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T17:36:36.458 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T17:36:36.458 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T17:36:36.458 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T17:36:36.458 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T17:36:36.459 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T17:36:20.334436+0000", 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T17:36:36.460 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.765+0000 7f8561ded700 1 Processor -- start 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.765+0000 7f8561ded700 1 -- start start 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.765+0000 7f8561ded700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f855c078ed0 con 0x7f855c106570 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.766+0000 7f855b7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f855c106570 0x7f855c108960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43186/0 (socket says 172.21.15.178:43186) 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.766+0000 7f855b7fe700 1 -- 172.21.15.178:0/2328754273 learned_addr learned my addr 172.21.15.178:0/2328754273 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.766+0000 7f855a7fc700 1 -- 172.21.15.178:0/2328754273 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641985684 0 0) 0x7f855c078ed0 con 0x7f855c106570 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.766+0000 7f855a7fc700 1 -- 172.21.15.178:0/2328754273 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85440035b0 con 0x7f855c106570 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.766+0000 7f855a7fc700 1 -- 172.21.15.178:0/2328754273 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 548400156 0 0) 0x7f85440035b0 con 0x7f855c106570 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.766+0000 7f855a7fc700 1 -- 172.21.15.178:0/2328754273 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c10b090 con 0x7f855c106570 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.766+0000 7f855a7fc700 1 -- 172.21.15.178:0/2328754273 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f854c001580 con 0x7f855c106570 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.767+0000 7f855a7fc700 1 -- 172.21.15.178:0/2328754273 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f854c001ce0 con 0x7f855c106570 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.767+0000 7f855a7fc700 1 -- 172.21.15.178:0/2328754273 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f854c005960 con 0x7f855c106570 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.767+0000 7f8561ded700 1 -- 172.21.15.178:0/2328754273 >> v1:172.21.15.178:6789/0 conn(0x7f855c106570 legacy=0x7f855c108960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:36.461 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.767+0000 7f8561ded700 1 -- 172.21.15.178:0/2328754273 shutdown_connections 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.767+0000 7f8561ded700 1 -- 172.21.15.178:0/2328754273 >> 172.21.15.178:0/2328754273 conn(0x7f855c0fff80 msgr2=0x7f855c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.767+0000 7f8561ded700 1 -- 172.21.15.178:0/2328754273 shutdown_connections 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.767+0000 7f8561ded700 1 -- 172.21.15.178:0/2328754273 wait complete. 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.768+0000 7f8561ded700 1 Processor -- start 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.768+0000 7f8561ded700 1 -- start start 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.768+0000 7f8561ded700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f855c199b30 con 0x7f855c106570 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.768+0000 7f855b7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f855c106570 0x7f855c199420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43192/0 (socket says 172.21.15.178:43192) 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.768+0000 7f855b7fe700 1 -- 172.21.15.178:0/1590864658 learned_addr learned my addr 172.21.15.178:0/1590864658 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.768+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432555563 0 0) 0x7f855c199b30 con 0x7f855c106570 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.768+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85440054f0 con 0x7f855c106570 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.769+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3310772176 0 0) 0x7f85440054f0 con 0x7f855c106570 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.769+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f855c199b30 con 0x7f855c106570 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.769+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f854c004870 con 0x7f855c106570 2024-02-14T17:36:36.462 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.769+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1237078484 0 0) 0x7f855c199b30 con 0x7f855c106570 2024-02-14T17:36:36.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.769+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c199cc0 con 0x7f855c106570 2024-02-14T17:36:36.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.769+0000 7f8561ded700 1 -- 172.21.15.178:0/1590864658 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c199f70 con 0x7f855c106570 2024-02-14T17:36:36.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.769+0000 7f8561ded700 1 -- 172.21.15.178:0/1590864658 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c19dac0 con 0x7f855c106570 2024-02-14T17:36:36.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.769+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f854c004a40 con 0x7f855c106570 2024-02-14T17:36:36.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.769+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f854c006060 con 0x7f855c106570 2024-02-14T17:36:36.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.769+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7f854c006670 con 0x7f855c106570 2024-02-14T17:36:36.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.769+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 1943377556 0 0) 0x7f854c007d40 con 0x7f855c106570 2024-02-14T17:36:36.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.770+0000 7f8561ded700 1 -- 172.21.15.178:0/1590864658 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8548005150 con 0x7f855c106570 2024-02-14T17:36:36.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.772+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f854c005640 con 0x7f855c106570 2024-02-14T17:36:36.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.844+0000 7f8561ded700 1 -- 172.21.15.178:0/1590864658 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8548005400 con 0x7f855c106570 2024-02-14T17:36:36.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.845+0000 7f8558ff9700 1 -- 172.21.15.178:0/1590864658 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (unknown 4201413639 0 916284147) 0x7f854c005880 con 0x7f855c106570 2024-02-14T17:36:36.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.845+0000 7f8561ded700 1 -- 172.21.15.178:0/1590864658 >> v1:172.21.15.178:6789/0 conn(0x7f855c106570 legacy=0x7f855c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:36.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.846+0000 7f8561ded700 1 -- 172.21.15.178:0/1590864658 shutdown_connections 2024-02-14T17:36:36.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.846+0000 7f8561ded700 1 -- 172.21.15.178:0/1590864658 >> 172.21.15.178:0/1590864658 conn(0x7f855c0fff80 msgr2=0x7f855c190250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:36.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.846+0000 7f8561ded700 1 -- 172.21.15.178:0/1590864658 shutdown_connections 2024-02-14T17:36:36.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:35.846+0000 7f8561ded700 1 -- 172.21.15.178:0/1590864658 wait complete. 2024-02-14T17:36:36.464 INFO:teuthology.orchestra.run.smithi178.stdout:mgr not available, waiting (1/15)... 2024-02-14T17:36:36.523 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:36 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:36.424+0000 7fb53e5cc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:36:36.523 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:36 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:36.521+0000 7fb53e5cc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:36:36.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:36 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:36.660+0000 7fb53e5cc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:36:36.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:36 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:36.753+0000 7fb53e5cc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:36:37.094 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:36 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:36.834+0000 7fb53e5cc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:36:37.094 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:37 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:37.092+0000 7fb53e5cc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:36:37.377 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:37 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:37.200+0000 7fb53e5cc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:36:37.377 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:37 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:37.283+0000 7fb53e5cc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:36:37.377 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:37 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:37.375+0000 7fb53e5cc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:36:37.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:37 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:37.549+0000 7fb53e5cc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:36:38.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:38 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:38.520+0000 7fb53e5cc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:36:38.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:38 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:38.651+0000 7fb53e5cc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:36:39.333 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:38.985+0000 7fb53e5cc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:36:39.333 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:39.074+0000 7fb53e5cc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:36:39.333 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:39.269+0000 7fb53e5cc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:36:39.728 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:39.453+0000 7fb53e5cc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:36:40.022 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3215139569' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T17:36:40.023 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:39.744+0000 7fb53e5cc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:36:40.023 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:40 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:40.020+0000 7fb53e5cc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:36:40.332 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:40 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:40.258+0000 7fb53e5cc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:36:40.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:40.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:36:40.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "fsid": "6e1588ec-cb5f-11ee-95b9-87774f69a715", 2024-02-14T17:36:40.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T17:36:40.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 0 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T17:36:40.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:40.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T17:36:40.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:40.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T17:36:40.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T17:36:40.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T17:36:40.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T17:36:40.463 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:36:40.464 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T17:36:20.334436+0000", 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.834+0000 7fc5dc04c700 1 Processor -- start 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.835+0000 7fc5dc04c700 1 -- start start 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.835+0000 7fc5dc04c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc5d4078e50 con 0x7fc5d41043e0 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.835+0000 7fc5d9de8700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc5d41043e0 0x7fc5d4104800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43208/0 (socket says 172.21.15.178:43208) 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.835+0000 7fc5d9de8700 1 -- 172.21.15.178:0/2741463158 learned_addr learned my addr 172.21.15.178:0/2741463158 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.835+0000 7fc5d8de6700 1 -- 172.21.15.178:0/2741463158 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1781473465 0 0) 0x7fc5d4078e50 con 0x7fc5d41043e0 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.835+0000 7fc5d8de6700 1 -- 172.21.15.178:0/2741463158 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c00035b0 con 0x7fc5d41043e0 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.836+0000 7fc5d8de6700 1 -- 172.21.15.178:0/2741463158 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3397884833 0 0) 0x7fc5c00035b0 con 0x7fc5d41043e0 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.836+0000 7fc5d8de6700 1 -- 172.21.15.178:0/2741463158 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d4105f20 con 0x7fc5d41043e0 2024-02-14T17:36:40.465 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.836+0000 7fc5d8de6700 1 -- 172.21.15.178:0/2741463158 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc5d0001580 con 0x7fc5d41043e0 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.836+0000 7fc5d8de6700 1 -- 172.21.15.178:0/2741463158 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7fc5d0001ce0 con 0x7fc5d41043e0 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.836+0000 7fc5d8de6700 1 -- 172.21.15.178:0/2741463158 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc5d0005960 con 0x7fc5d41043e0 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.836+0000 7fc5dc04c700 1 -- 172.21.15.178:0/2741463158 >> v1:172.21.15.178:6789/0 conn(0x7fc5d41043e0 legacy=0x7fc5d4104800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.837+0000 7fc5dc04c700 1 -- 172.21.15.178:0/2741463158 shutdown_connections 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.837+0000 7fc5dc04c700 1 -- 172.21.15.178:0/2741463158 >> 172.21.15.178:0/2741463158 conn(0x7fc5d40fff80 msgr2=0x7fc5d41023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.837+0000 7fc5dc04c700 1 -- 172.21.15.178:0/2741463158 shutdown_connections 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.837+0000 7fc5dc04c700 1 -- 172.21.15.178:0/2741463158 wait complete. 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.837+0000 7fc5dc04c700 1 Processor -- start 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.837+0000 7fc5dc04c700 1 -- start start 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5dc04c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc5d419bd30 con 0x7fc5d41043e0 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5d9de8700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc5d41043e0 0x7fc5d419b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43216/0 (socket says 172.21.15.178:43216) 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5d9de8700 1 -- 172.21.15.178:0/3215139569 learned_addr learned my addr 172.21.15.178:0/3215139569 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173861852 0 0) 0x7fc5d419bd30 con 0x7fc5d41043e0 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c00054f0 con 0x7fc5d41043e0 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2759430304 0 0) 0x7fc5c00054f0 con 0x7fc5d41043e0 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5d419bd30 con 0x7fc5d41043e0 2024-02-14T17:36:40.466 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc5d0004870 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1680721659 0 0) 0x7fc5d419bd30 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d419bec0 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5dc04c700 1 -- 172.21.15.178:0/3215139569 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d419c170 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.838+0000 7fc5dc04c700 1 -- 172.21.15.178:0/3215139569 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d419fcc0 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.839+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7fc5d0001190 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.839+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc5d0006130 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.839+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7fc5d0006720 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.839+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 1943377556 0 0) 0x7fc5d0007da0 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.839+0000 7fc5dc04c700 1 -- 172.21.15.178:0/3215139569 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b8005150 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.841+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fc5d0004a40 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.914+0000 7fc5dc04c700 1 -- 172.21.15.178:0/3215139569 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc5b8005400 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.915+0000 7fc5caffd700 1 -- 172.21.15.178:0/3215139569 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (unknown 4201413639 0 148023024) 0x7fc5d0004ca0 con 0x7fc5d41043e0 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.916+0000 7fc5dc04c700 1 -- 172.21.15.178:0/3215139569 >> v1:172.21.15.178:6789/0 conn(0x7fc5d41043e0 legacy=0x7fc5d419b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.916+0000 7fc5dc04c700 1 -- 172.21.15.178:0/3215139569 shutdown_connections 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.916+0000 7fc5dc04c700 1 -- 172.21.15.178:0/3215139569 >> 172.21.15.178:0/3215139569 conn(0x7fc5d40fff80 msgr2=0x7fc5d41023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:40.467 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.916+0000 7fc5dc04c700 1 -- 172.21.15.178:0/3215139569 shutdown_connections 2024-02-14T17:36:40.468 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:39.916+0000 7fc5dc04c700 1 -- 172.21.15.178:0/3215139569 wait complete. 2024-02-14T17:36:40.468 INFO:teuthology.orchestra.run.smithi178.stdout:mgr not available, waiting (2/15)... 2024-02-14T17:36:41.303 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:41 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:41.061+0000 7fb53e5cc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:36:41.983 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:41 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:41.658+0000 7fb53e5cc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:36:41.983 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:41 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:41.760+0000 7fb53e5cc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:36:41.983 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:41 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:41.981+0000 7fb53e5cc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:36:42.257 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: Activating manager daemon y 2024-02-14T17:36:42.257 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: mgrmap e2: y(active, starting, since 0.00386319s) 2024-02-14T17:36:42.257 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:36:42.257 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:36:42.257 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:36:42.257 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:36:42.257 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:36:42.257 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:36:42.257 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:36:42.257 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:36:42.257 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: Manager daemon y is now available 2024-02-14T17:36:42.258 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:36:42.258 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:36:42.258 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' 2024-02-14T17:36:42.258 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' 2024-02-14T17:36:42.258 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:42 smithi178 ceph-mon[30321]: from='mgr.14100 v1:172.21.15.178:0/3459919107' entity='mgr.y' 2024-02-14T17:36:43.626 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:43.628 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:36:43.628 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "fsid": "6e1588ec-cb5f-11ee-95b9-87774f69a715", 2024-02-14T17:36:43.628 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 0 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T17:36:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T17:36:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T17:36:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T17:36:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T17:36:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T17:36:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T17:36:43.638 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:36:43.639 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T17:36:20.334436+0000", 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.950+0000 7f244ac7c700 1 Processor -- start 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.950+0000 7f244ac7c700 1 -- start start 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.950+0000 7f244ac7c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2444108e40 con 0x7f2444104a10 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.950+0000 7f2448a18700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2444104a10 0x7f2444104e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43290/0 (socket says 172.21.15.178:43290) 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.950+0000 7f2448a18700 1 -- 172.21.15.178:0/2923308691 learned_addr learned my addr 172.21.15.178:0/2923308691 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:43.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.950+0000 7f24437fe700 1 -- 172.21.15.178:0/2923308691 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782348843 0 0) 0x7f2444108e40 con 0x7f2444104a10 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.950+0000 7f24437fe700 1 -- 172.21.15.178:0/2923308691 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24340035b0 con 0x7f2444104a10 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.951+0000 7f24437fe700 1 -- 172.21.15.178:0/2923308691 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1669124644 0 0) 0x7f24340035b0 con 0x7f2444104a10 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.951+0000 7f24437fe700 1 -- 172.21.15.178:0/2923308691 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2444109fe0 con 0x7f2444104a10 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.951+0000 7f24437fe700 1 -- 172.21.15.178:0/2923308691 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2438001580 con 0x7f2444104a10 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.951+0000 7f24437fe700 1 -- 172.21.15.178:0/2923308691 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f2438001ce0 con 0x7f2444104a10 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.951+0000 7f24437fe700 1 -- 172.21.15.178:0/2923308691 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2438005960 con 0x7f2444104a10 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.951+0000 7f244ac7c700 1 -- 172.21.15.178:0/2923308691 >> v1:172.21.15.178:6789/0 conn(0x7f2444104a10 legacy=0x7f2444104e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.951+0000 7f244ac7c700 1 -- 172.21.15.178:0/2923308691 shutdown_connections 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.951+0000 7f244ac7c700 1 -- 172.21.15.178:0/2923308691 >> 172.21.15.178:0/2923308691 conn(0x7f24440fffd0 msgr2=0x7f24441023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.951+0000 7f244ac7c700 1 -- 172.21.15.178:0/2923308691 shutdown_connections 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.951+0000 7f244ac7c700 1 -- 172.21.15.178:0/2923308691 wait complete. 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.952+0000 7f244ac7c700 1 Processor -- start 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.952+0000 7f244ac7c700 1 -- start start 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.952+0000 7f244ac7c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2444199b30 con 0x7f2444104a10 2024-02-14T17:36:43.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.952+0000 7f2448a18700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2444104a10 0x7f2444199420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43292/0 (socket says 172.21.15.178:43292) 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.952+0000 7f2448a18700 1 -- 172.21.15.178:0/154146084 learned_addr learned my addr 172.21.15.178:0/154146084 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.952+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008612545 0 0) 0x7f2444199b30 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.952+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24340054f0 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.953+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 268844933 0 0) 0x7f24340054f0 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.953+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2444199b30 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.953+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2438005940 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.953+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3022837260 0 0) 0x7f2444199b30 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.953+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2444199cc0 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.953+0000 7f244ac7c700 1 -- 172.21.15.178:0/154146084 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2444199f70 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.953+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f2438001b20 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.953+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f24380061e0 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.953+0000 7f244ac7c700 1 -- 172.21.15.178:0/154146084 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f244419dac0 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.954+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 2) v1 ==== 45688+0+0 (unknown 3410808975 0 0) 0x7f2438011730 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.954+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 1943377556 0 0) 0x7f2438046e70 con 0x7f2444104a10 2024-02-14T17:36:43.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.954+0000 7f244ac7c700 1 -- 172.21.15.178:0/154146084 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24440632a0 con 0x7f2444104a10 2024-02-14T17:36:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.956+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f2438016a80 con 0x7f2444104a10 2024-02-14T17:36:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:42.998+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mgrmap(e 3) v1 ==== 45780+0+0 (unknown 3411695198 0 0) 0x7f243802a860 con 0x7f2444104a10 2024-02-14T17:36:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:43.029+0000 7f244ac7c700 1 -- 172.21.15.178:0/154146084 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f244419def0 con 0x7f2444104a10 2024-02-14T17:36:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:43.029+0000 7f2441ffb700 1 -- 172.21.15.178:0/154146084 <== mon.0 v1:172.21.15.178:6789/0 11 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (unknown 4201413639 0 3099682486) 0x7f24380170e0 con 0x7f2444104a10 2024-02-14T17:36:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:43.030+0000 7f244ac7c700 1 -- 172.21.15.178:0/154146084 >> v1:172.21.15.178:6800/1774918016 conn(0x7f243403ada0 legacy=0x7f243403d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:43.030+0000 7f244ac7c700 1 -- 172.21.15.178:0/154146084 >> v1:172.21.15.178:6789/0 conn(0x7f2444104a10 legacy=0x7f2444199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:43.030+0000 7f244ac7c700 1 -- 172.21.15.178:0/154146084 shutdown_connections 2024-02-14T17:36:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:43.030+0000 7f244ac7c700 1 -- 172.21.15.178:0/154146084 >> 172.21.15.178:0/154146084 conn(0x7f24440fffd0 msgr2=0x7f2444078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:43.030+0000 7f244ac7c700 1 -- 172.21.15.178:0/154146084 shutdown_connections 2024-02-14T17:36:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:43.030+0000 7f244ac7c700 1 -- 172.21.15.178:0/154146084 wait complete. 2024-02-14T17:36:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:mgr is available 2024-02-14T17:36:44.270 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:43 smithi178 ceph-mon[30321]: mgrmap e3: y(active, since 1.00606s) 2024-02-14T17:36:44.270 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/154146084' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T17:36:45.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2715236110' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-14T17:36:45.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:45.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [global] 2024-02-14T17:36:45.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout fsid = 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:36:45.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-14T17:36:45.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-14T17:36:45.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-14T17:36:45.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-14T17:36:45.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [osd] 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.069+0000 7f98e4e6a700 1 Processor -- start 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.069+0000 7f98e4e6a700 1 -- start start 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.069+0000 7f98e4e6a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f98e0078e30 con 0x7f98e0104400 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.070+0000 7f98de59c700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f98e0104400 0x7f98e0104820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39862/0 (socket says 172.21.15.178:39862) 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.070+0000 7f98de59c700 1 -- 172.21.15.178:0/1250753499 learned_addr learned my addr 172.21.15.178:0/1250753499 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.070+0000 7f98dd59a700 1 -- 172.21.15.178:0/1250753499 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693242151 0 0) 0x7f98e0078e30 con 0x7f98e0104400 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.070+0000 7f98dd59a700 1 -- 172.21.15.178:0/1250753499 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c80035b0 con 0x7f98e0104400 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.070+0000 7f98dd59a700 1 -- 172.21.15.178:0/1250753499 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3867829584 0 0) 0x7f98c80035b0 con 0x7f98e0104400 2024-02-14T17:36:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.070+0000 7f98dd59a700 1 -- 172.21.15.178:0/1250753499 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e0105f40 con 0x7f98e0104400 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.070+0000 7f98dd59a700 1 -- 172.21.15.178:0/1250753499 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f98d8001900 con 0x7f98e0104400 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.070+0000 7f98dd59a700 1 -- 172.21.15.178:0/1250753499 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f98d8002060 con 0x7f98e0104400 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.070+0000 7f98dd59a700 1 -- 172.21.15.178:0/1250753499 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f98d8005ce0 con 0x7f98e0104400 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.071+0000 7f98e4e6a700 1 -- 172.21.15.178:0/1250753499 >> v1:172.21.15.178:6789/0 conn(0x7f98e0104400 legacy=0x7f98e0104820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.071+0000 7f98e4e6a700 1 -- 172.21.15.178:0/1250753499 shutdown_connections 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.071+0000 7f98e4e6a700 1 -- 172.21.15.178:0/1250753499 >> 172.21.15.178:0/1250753499 conn(0x7f98e00fff80 msgr2=0x7f98e01023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.071+0000 7f98e4e6a700 1 -- 172.21.15.178:0/1250753499 shutdown_connections 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.071+0000 7f98e4e6a700 1 -- 172.21.15.178:0/1250753499 wait complete. 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.071+0000 7f98e4e6a700 1 Processor -- start 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98e4e6a700 1 -- start start 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98e4e6a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f98e019bd30 con 0x7f98e0104400 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98de59c700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f98e0104400 0x7f98e019b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39866/0 (socket says 172.21.15.178:39866) 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98de59c700 1 -- 172.21.15.178:0/2715236110 learned_addr learned my addr 172.21.15.178:0/2715236110 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:45.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246732400 0 0) 0x7f98e019bd30 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c80054f0 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1887823705 0 0) 0x7f98c80054f0 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f98e019bd30 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f98d8004bf0 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4092792980 0 0) 0x7f98e019bd30 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e019bec0 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98e4e6a700 1 -- 172.21.15.178:0/2715236110 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e019c170 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98e4e6a700 1 -- 172.21.15.178:0/2715236110 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e019fcc0 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f98d8004dc0 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.072+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f98d80064b0 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.073+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 3) v1 ==== 45780+0+0 (unknown 3411695198 0 0) 0x7f98d8011910 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.073+0000 7f98e4e6a700 1 -- 172.21.15.178:0/2715236110 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c0005120 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.073+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 1943377556 0 0) 0x7f98d8047390 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.078+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f98d8013120 con 0x7f98e0104400 2024-02-14T17:36:45.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.268+0000 7f98e4e6a700 1 -- 172.21.15.178:0/2715236110 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f98c0005950 con 0x7f98e0104400 2024-02-14T17:36:45.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.269+0000 7f98cf7fe700 1 -- 172.21.15.178:0/2715236110 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+282 (unknown 1187553405 0 2781480623) 0x7f98d8015200 con 0x7f98e0104400 2024-02-14T17:36:45.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.271+0000 7f98e4e6a700 1 -- 172.21.15.178:0/2715236110 >> v1:172.21.15.178:6800/1774918016 conn(0x7f98c8039ff0 legacy=0x7f98c803c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:45.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.271+0000 7f98e4e6a700 1 -- 172.21.15.178:0/2715236110 >> v1:172.21.15.178:6789/0 conn(0x7f98e0104400 legacy=0x7f98e019b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:45.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.271+0000 7f98e4e6a700 1 -- 172.21.15.178:0/2715236110 shutdown_connections 2024-02-14T17:36:45.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.271+0000 7f98e4e6a700 1 -- 172.21.15.178:0/2715236110 >> 172.21.15.178:0/2715236110 conn(0x7f98e00fff80 msgr2=0x7f98e01023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:45.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.271+0000 7f98e4e6a700 1 -- 172.21.15.178:0/2715236110 shutdown_connections 2024-02-14T17:36:45.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:44.271+0000 7f98e4e6a700 1 -- 172.21.15.178:0/2715236110 wait complete. 2024-02-14T17:36:45.368 INFO:teuthology.orchestra.run.smithi178.stdout:Enabling cephadm module... 2024-02-14T17:36:46.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:46 smithi178 ceph-mon[30321]: mgrmap e4: y(active, since 3s) 2024-02-14T17:36:47.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1991445683' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-14T17:36:47.333 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:47 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ignoring --setuser ceph since I am not root 2024-02-14T17:36:47.333 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:47 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ignoring --setgroup ceph since I am not root 2024-02-14T17:36:47.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.820+0000 7f0c9d243700 1 Processor -- start 2024-02-14T17:36:47.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.820+0000 7f0c9d243700 1 -- start start 2024-02-14T17:36:47.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.820+0000 7f0c9d243700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c980796c0 con 0x7f0c981043f0 2024-02-14T17:36:47.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.820+0000 7f0c96d9d700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f0c981043f0 0x7f0c98104810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39880/0 (socket says 172.21.15.178:39880) 2024-02-14T17:36:47.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.820+0000 7f0c96d9d700 1 -- 172.21.15.178:0/2943117975 learned_addr learned my addr 172.21.15.178:0/2943117975 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:47.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.821+0000 7f0c9659c700 1 -- 172.21.15.178:0/2943117975 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689720648 0 0) 0x7f0c980796c0 con 0x7f0c981043f0 2024-02-14T17:36:47.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.821+0000 7f0c9659c700 1 -- 172.21.15.178:0/2943117975 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c840035b0 con 0x7f0c981043f0 2024-02-14T17:36:47.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.821+0000 7f0c9659c700 1 -- 172.21.15.178:0/2943117975 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 281007152 0 0) 0x7f0c840035b0 con 0x7f0c981043f0 2024-02-14T17:36:47.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.821+0000 7f0c9659c700 1 -- 172.21.15.178:0/2943117975 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c98105f30 con 0x7f0c981043f0 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.821+0000 7f0c9659c700 1 -- 172.21.15.178:0/2943117975 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f0c88001580 con 0x7f0c981043f0 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.821+0000 7f0c9659c700 1 -- 172.21.15.178:0/2943117975 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f0c88001c10 con 0x7f0c981043f0 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.821+0000 7f0c9659c700 1 -- 172.21.15.178:0/2943117975 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f0c880057b0 con 0x7f0c981043f0 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.822+0000 7f0c9d243700 1 -- 172.21.15.178:0/2943117975 >> v1:172.21.15.178:6789/0 conn(0x7f0c981043f0 legacy=0x7f0c98104810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.822+0000 7f0c9d243700 1 -- 172.21.15.178:0/2943117975 shutdown_connections 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.822+0000 7f0c9d243700 1 -- 172.21.15.178:0/2943117975 >> 172.21.15.178:0/2943117975 conn(0x7f0c980fff70 msgr2=0x7f0c981023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.822+0000 7f0c9d243700 1 -- 172.21.15.178:0/2943117975 shutdown_connections 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.822+0000 7f0c9d243700 1 -- 172.21.15.178:0/2943117975 wait complete. 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.822+0000 7f0c9d243700 1 Processor -- start 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.822+0000 7f0c9d243700 1 -- start start 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.822+0000 7f0c9d243700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c9819bd00 con 0x7f0c981043f0 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c96d9d700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f0c981043f0 0x7f0c9819b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39884/0 (socket says 172.21.15.178:39884) 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c96d9d700 1 -- 172.21.15.178:0/1991445683 learned_addr learned my addr 172.21.15.178:0/1991445683 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447769252 0 0) 0x7f0c9819bd00 con 0x7f0c981043f0 2024-02-14T17:36:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c84005540 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3437421283 0 0) 0x7f0c84005540 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c9819bd00 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f0c88001080 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3451606893 0 0) 0x7f0c9819bd00 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9819be90 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c9d243700 1 -- 172.21.15.178:0/1991445683 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9819c140 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c9d243700 1 -- 172.21.15.178:0/1991445683 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9819fc90 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f0c88004900 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.823+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f0c88006240 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.824+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 4) v1 ==== 45886+0+0 (unknown 4085284564 0 0) 0x7f0c880117d0 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.824+0000 7f0c9d243700 1 -- 172.21.15.178:0/1991445683 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c78005150 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.824+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 1943377556 0 0) 0x7f0c880473e0 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:45.829+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f0c88012ff0 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:46.077+0000 7f0c9d243700 1 -- 172.21.15.178:0/1991445683 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f0c78005400 con 0x7f0c981043f0 2024-02-14T17:36:47.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:47.005+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (unknown 2263024820 0 0) 0x7f0c880150d0 con 0x7f0c981043f0 2024-02-14T17:36:47.605 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:47.007+0000 7f0c94d99700 1 -- 172.21.15.178:0/1991445683 <== mon.0 v1:172.21.15.178:6789/0 11 ==== mgrmap(e 5) v1 ==== 45897+0+0 (unknown 2053839474 0 0) 0x7f0c88045dd0 con 0x7f0c981043f0 2024-02-14T17:36:47.605 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:47.009+0000 7f0c9d243700 1 -- 172.21.15.178:0/1991445683 >> v1:172.21.15.178:6800/1774918016 conn(0x7f0c8403a020 legacy=0x7f0c8403c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:47.605 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:47.009+0000 7f0c9d243700 1 -- 172.21.15.178:0/1991445683 >> v1:172.21.15.178:6789/0 conn(0x7f0c981043f0 legacy=0x7f0c9819b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:47.605 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:47.009+0000 7f0c9d243700 1 -- 172.21.15.178:0/1991445683 shutdown_connections 2024-02-14T17:36:47.605 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:47.009+0000 7f0c9d243700 1 -- 172.21.15.178:0/1991445683 >> 172.21.15.178:0/1991445683 conn(0x7f0c980fff70 msgr2=0x7f0c981023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:47.605 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:47.009+0000 7f0c9d243700 1 -- 172.21.15.178:0/1991445683 shutdown_connections 2024-02-14T17:36:47.605 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:47.009+0000 7f0c9d243700 1 -- 172.21.15.178:0/1991445683 wait complete. 2024-02-14T17:36:47.679 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:47 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:47.577+0000 7f137c1a9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:36:47.679 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:47 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:47.677+0000 7f137c1a9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:36:48.264 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1991445683' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-14T17:36:48.264 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:48 smithi178 ceph-mon[30321]: mgrmap e5: y(active, since 5s) 2024-02-14T17:36:48.568 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:48 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:48.342+0000 7f137c1a9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:36:48.568 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:48 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:48.437+0000 7f137c1a9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:36:48.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:48 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:48.582+0000 7f137c1a9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:36:48.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:48 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:48.678+0000 7f137c1a9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:36:48.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:48 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:48.761+0000 7f137c1a9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:36:48.884 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:36:48.884 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-14T17:36:48.884 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T17:36:48.884 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-14T17:36:48.884 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.057+0000 7f7927b73700 1 Processor -- start 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.057+0000 7f7927b73700 1 -- start start 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.057+0000 7f7927b73700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7920104b90 con 0x7f7920106080 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.057+0000 7f792590f700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f7920106080 0x7f79201064a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39914/0 (socket says 172.21.15.178:39914) 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.057+0000 7f792590f700 1 -- 172.21.15.178:0/1181172848 learned_addr learned my addr 172.21.15.178:0/1181172848 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.057+0000 7f792490d700 1 -- 172.21.15.178:0/1181172848 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293645710 0 0) 0x7f7920104b90 con 0x7f7920106080 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.058+0000 7f792490d700 1 -- 172.21.15.178:0/1181172848 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f790c0035b0 con 0x7f7920106080 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.058+0000 7f792490d700 1 -- 172.21.15.178:0/1181172848 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1615363158 0 0) 0x7f790c0035b0 con 0x7f7920106080 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.058+0000 7f792490d700 1 -- 172.21.15.178:0/1181172848 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7920104d20 con 0x7f7920106080 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.058+0000 7f792490d700 1 -- 172.21.15.178:0/1181172848 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f791c001580 con 0x7f7920106080 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.058+0000 7f792490d700 1 -- 172.21.15.178:0/1181172848 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f791c001c10 con 0x7f7920106080 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.058+0000 7f792490d700 1 -- 172.21.15.178:0/1181172848 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f791c0057b0 con 0x7f7920106080 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.058+0000 7f7927b73700 1 -- 172.21.15.178:0/1181172848 >> v1:172.21.15.178:6789/0 conn(0x7f7920106080 legacy=0x7f79201064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.059+0000 7f7927b73700 1 -- 172.21.15.178:0/1181172848 shutdown_connections 2024-02-14T17:36:48.885 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.059+0000 7f7927b73700 1 -- 172.21.15.178:0/1181172848 >> 172.21.15.178:0/1181172848 conn(0x7f79200ff700 msgr2=0x7f7920101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.059+0000 7f7927b73700 1 -- 172.21.15.178:0/1181172848 shutdown_connections 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.059+0000 7f7927b73700 1 -- 172.21.15.178:0/1181172848 wait complete. 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.059+0000 7f7927b73700 1 Processor -- start 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.059+0000 7f7927b73700 1 -- start start 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.059+0000 7f7927b73700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f792019bd00 con 0x7f7920106080 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.059+0000 7f792590f700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f7920106080 0x7f792019b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39922/0 (socket says 172.21.15.178:39922) 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.059+0000 7f792590f700 1 -- 172.21.15.178:0/3919182974 learned_addr learned my addr 172.21.15.178:0/3919182974 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.060+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920607209 0 0) 0x7f792019bd00 con 0x7f7920106080 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.060+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f790c005540 con 0x7f7920106080 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.060+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1385689956 0 0) 0x7f790c005540 con 0x7f7920106080 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.060+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f792019bd00 con 0x7f7920106080 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.060+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f791c001080 con 0x7f7920106080 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.060+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2925767953 0 0) 0x7f792019bd00 con 0x7f7920106080 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.060+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792019be90 con 0x7f7920106080 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.060+0000 7f7927b73700 1 -- 172.21.15.178:0/3919182974 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792019c140 con 0x7f7920106080 2024-02-14T17:36:48.886 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.060+0000 7f7927b73700 1 -- 172.21.15.178:0/3919182974 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f792019fc90 con 0x7f7920106080 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.060+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f791c004900 con 0x7f7920106080 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.060+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f791c006240 con 0x7f7920106080 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.061+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 5) v1 ==== 45897+0+0 (unknown 2053839474 0 0) 0x7f791c0117e0 con 0x7f7920106080 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.061+0000 7f7927b73700 1 -- 172.21.15.178:0/3919182974 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7904005150 con 0x7f7920106080 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.061+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 1943377556 0 0) 0x7f791c048260 con 0x7f7920106080 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.061+0000 7f792510e700 1 -- 172.21.15.178:0/3919182974 >> v1:172.21.15.178:6800/1774918016 conn(0x7f790c03a070 legacy=0x7f790c03c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1774918016 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.066+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f791c013050 con 0x7f7920106080 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.261+0000 7f792510e700 1 -- 172.21.15.178:0/3919182974 >> v1:172.21.15.178:6800/1774918016 conn(0x7f790c03a070 legacy=0x7f790c03c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1774918016 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.321+0000 7f7927b73700 1 -- 172.21.15.178:0/3919182974 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7904005c10 con 0x7f7920106080 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.321+0000 7f7916ffd700 1 -- 172.21.15.178:0/3919182974 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (unknown 3768197548 0 15966916) 0x7f791c015130 con 0x7f7920106080 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.323+0000 7f7927b73700 1 -- 172.21.15.178:0/3919182974 >> v1:172.21.15.178:6800/1774918016 conn(0x7f790c03a070 legacy=0x7f790c03c530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.323+0000 7f7927b73700 1 -- 172.21.15.178:0/3919182974 >> v1:172.21.15.178:6789/0 conn(0x7f7920106080 legacy=0x7f792019b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.323+0000 7f7927b73700 1 -- 172.21.15.178:0/3919182974 shutdown_connections 2024-02-14T17:36:48.887 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.323+0000 7f7927b73700 1 -- 172.21.15.178:0/3919182974 >> 172.21.15.178:0/3919182974 conn(0x7f79200ff700 msgr2=0x7f7920101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:48.888 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.324+0000 7f7927b73700 1 -- 172.21.15.178:0/3919182974 shutdown_connections 2024-02-14T17:36:48.888 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:48.324+0000 7f7927b73700 1 -- 172.21.15.178:0/3919182974 wait complete. 2024-02-14T17:36:48.888 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for the mgr to restart... 2024-02-14T17:36:48.888 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mgr epoch 5... 2024-02-14T17:36:49.132 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3919182974' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-14T17:36:49.132 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:49 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:49.032+0000 7f137c1a9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:36:49.383 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:49 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:49.147+0000 7f137c1a9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:36:49.383 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:49 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:49.237+0000 7f137c1a9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:36:49.383 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:49 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:49.333+0000 7f137c1a9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:36:49.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:49 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:49.514+0000 7f137c1a9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:36:50.792 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:50 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:50.466+0000 7f137c1a9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:36:50.792 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:50 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:50.594+0000 7f137c1a9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:36:51.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:50 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:50.925+0000 7f137c1a9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:36:51.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:51 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:51.014+0000 7f137c1a9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:36:51.582 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:51 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:51.207+0000 7f137c1a9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:36:51.583 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:51 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:51.390+0000 7f137c1a9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:36:51.941 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:51 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:51.673+0000 7f137c1a9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:36:51.941 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:51 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:51.938+0000 7f137c1a9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:36:52.320 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:52 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:52.172+0000 7f137c1a9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:36:53.333 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:52 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:52.968+0000 7f137c1a9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:36:53.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:53.534+0000 7f137c1a9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:36:53.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:53.632+0000 7f137c1a9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:36:54.332 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:36:53.848+0000 7f137c1a9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: Active manager daemon y restarted 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: Activating manager daemon y 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: osdmap e2: 0 total, 0 up, 0 in 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: mgrmap e6: y(active, starting, since 0.00313078s) 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: Manager daemon y is now available 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:36:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:36:54.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:36:54.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:36:54.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:53 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:36:55.175 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:54 smithi178 ceph-mon[30321]: Found migration_current of "None". Setting to last migration. 2024-02-14T17:36:55.175 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:54 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:36:55.176 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:54 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:36:55.176 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:54 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:36:55.176 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:54 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:36:55.176 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:54 smithi178 ceph-mon[30321]: mgrmap e7: y(active, since 1.00515s) 2024-02-14T17:36:55.619 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:36:55.619 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-14T17:36:55.619 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-14T17:36:55.619 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:36:55.619 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.380+0000 7fe6a10d7700 1 Processor -- start 2024-02-14T17:36:55.620 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.380+0000 7fe6a10d7700 1 -- start start 2024-02-14T17:36:55.620 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.380+0000 7fe6a10d7700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe69c109f30 con 0x7fe69c105b00 2024-02-14T17:36:55.620 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.380+0000 7fe69ad9d700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe69c105b00 0x7fe69c105f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39928/0 (socket says 172.21.15.178:39928) 2024-02-14T17:36:55.620 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.380+0000 7fe69ad9d700 1 -- 172.21.15.178:0/1451570503 learned_addr learned my addr 172.21.15.178:0/1451570503 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:55.620 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.381+0000 7fe699d9b700 1 -- 172.21.15.178:0/1451570503 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531470582 0 0) 0x7fe69c109f30 con 0x7fe69c105b00 2024-02-14T17:36:55.620 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.381+0000 7fe699d9b700 1 -- 172.21.15.178:0/1451570503 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6840035b0 con 0x7fe69c105b00 2024-02-14T17:36:55.620 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.381+0000 7fe699d9b700 1 -- 172.21.15.178:0/1451570503 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2561121731 0 0) 0x7fe6840035b0 con 0x7fe69c105b00 2024-02-14T17:36:55.620 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.381+0000 7fe699d9b700 1 -- 172.21.15.178:0/1451570503 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c10b0d0 con 0x7fe69c105b00 2024-02-14T17:36:55.620 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.381+0000 7fe699d9b700 1 -- 172.21.15.178:0/1451570503 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fe690001580 con 0x7fe69c105b00 2024-02-14T17:36:55.620 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.381+0000 7fe699d9b700 1 -- 172.21.15.178:0/1451570503 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7fe690001ce0 con 0x7fe69c105b00 2024-02-14T17:36:55.620 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.381+0000 7fe699d9b700 1 -- 172.21.15.178:0/1451570503 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fe690005960 con 0x7fe69c105b00 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.382+0000 7fe6a10d7700 1 -- 172.21.15.178:0/1451570503 >> v1:172.21.15.178:6789/0 conn(0x7fe69c105b00 legacy=0x7fe69c105f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.382+0000 7fe6a10d7700 1 -- 172.21.15.178:0/1451570503 shutdown_connections 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.382+0000 7fe6a10d7700 1 -- 172.21.15.178:0/1451570503 >> 172.21.15.178:0/1451570503 conn(0x7fe69c1010c0 msgr2=0x7fe69c1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.382+0000 7fe6a10d7700 1 -- 172.21.15.178:0/1451570503 shutdown_connections 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.382+0000 7fe6a10d7700 1 -- 172.21.15.178:0/1451570503 wait complete. 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.383+0000 7fe6a10d7700 1 Processor -- start 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.383+0000 7fe6a10d7700 1 -- start start 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.383+0000 7fe6a10d7700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe69c199b10 con 0x7fe69c105b00 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.384+0000 7fe69ad9d700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe69c105b00 0x7fe69c199400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39934/0 (socket says 172.21.15.178:39934) 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.384+0000 7fe69ad9d700 1 -- 172.21.15.178:0/2307448660 learned_addr learned my addr 172.21.15.178:0/2307448660 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.384+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374158927 0 0) 0x7fe69c199b10 con 0x7fe69c105b00 2024-02-14T17:36:55.621 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.384+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6840054f0 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.384+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1953264757 0 0) 0x7fe6840054f0 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.384+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe69c199b10 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.384+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fe690005940 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.384+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2494296200 0 0) 0x7fe69c199b10 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.384+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c199ca0 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.384+0000 7fe6a10d7700 1 -- 172.21.15.178:0/2307448660 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69c199f50 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.384+0000 7fe6a10d7700 1 -- 172.21.15.178:0/2307448660 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69c19daa0 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.385+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7fe690001b20 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.385+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fe6900060b0 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.385+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 5) v1 ==== 45897+0+0 (unknown 2053839474 0 0) 0x7fe690006230 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.386+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 --> v1:172.21.15.178:6800/1774918016 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe68403de90 con 0x7fe68403a1c0 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.386+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 1943377556 0 0) 0x7fe690048060 con 0x7fe69c105b00 2024-02-14T17:36:55.622 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.386+0000 7fe69a59c700 1 -- 172.21.15.178:0/2307448660 >> v1:172.21.15.178:6800/1774918016 conn(0x7fe68403a1c0 legacy=0x7fe68403c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1774918016 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.587+0000 7fe69a59c700 1 -- 172.21.15.178:0/2307448660 >> v1:172.21.15.178:6800/1774918016 conn(0x7fe68403a1c0 legacy=0x7fe68403c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1774918016 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:49.988+0000 7fe69a59c700 1 -- 172.21.15.178:0/2307448660 >> v1:172.21.15.178:6800/1774918016 conn(0x7fe68403a1c0 legacy=0x7fe68403c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1774918016 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:50.789+0000 7fe69a59c700 1 -- 172.21.15.178:0/2307448660 >> v1:172.21.15.178:6800/1774918016 conn(0x7fe68403a1c0 legacy=0x7fe68403c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1774918016 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:52.390+0000 7fe69a59c700 1 -- 172.21.15.178:0/2307448660 >> v1:172.21.15.178:6800/1774918016 conn(0x7fe68403a1c0 legacy=0x7fe68403c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1774918016 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:53.862+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mgrmap(e 6) v1 ==== 45699+0+0 (unknown 821756624 0 0) 0x7fe690005690 con 0x7fe69c105b00 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:53.863+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 >> v1:172.21.15.178:6800/1774918016 conn(0x7fe68403a1c0 legacy=0x7fe68403c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:54.865+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mgrmap(e 7) v1 ==== 45791+0+0 (unknown 1721836910 0 0) 0x7fe690048bf0 con 0x7fe69c105b00 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:54.865+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 --> v1:172.21.15.178:6800/366374299 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe68403de90 con 0x7fe68403eb90 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:54.866+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mgr.14118 v1:172.21.15.178:6800/366374299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (unknown 0 0 4109177037) 0x7fe68403de90 con 0x7fe68403eb90 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:54.872+0000 7fe6a10d7700 1 -- 172.21.15.178:0/2307448660 --> v1:172.21.15.178:6800/366374299 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe688002630 con 0x7fe68403eb90 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:54.873+0000 7fe683fff700 1 -- 172.21.15.178:0/2307448660 <== mgr.14118 v1:172.21.15.178:6800/366374299 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (unknown 0 0 96372106) 0x7fe688002630 con 0x7fe68403eb90 2024-02-14T17:36:55.623 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:54.873+0000 7fe6a10d7700 1 -- 172.21.15.178:0/2307448660 >> v1:172.21.15.178:6800/366374299 conn(0x7fe68403eb90 legacy=0x7fe684040f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:55.624 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:54.873+0000 7fe6a10d7700 1 -- 172.21.15.178:0/2307448660 >> v1:172.21.15.178:6789/0 conn(0x7fe69c105b00 legacy=0x7fe69c199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:55.624 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:54.873+0000 7fe6a10d7700 1 -- 172.21.15.178:0/2307448660 shutdown_connections 2024-02-14T17:36:55.624 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:54.873+0000 7fe6a10d7700 1 -- 172.21.15.178:0/2307448660 >> 172.21.15.178:0/2307448660 conn(0x7fe69c1010c0 msgr2=0x7fe69c1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:55.624 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:54.873+0000 7fe6a10d7700 1 -- 172.21.15.178:0/2307448660 shutdown_connections 2024-02-14T17:36:55.624 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:54.873+0000 7fe6a10d7700 1 -- 172.21.15.178:0/2307448660 wait complete. 2024-02-14T17:36:55.624 INFO:teuthology.orchestra.run.smithi178.stdout:mgr epoch 5 is available 2024-02-14T17:36:55.624 INFO:teuthology.orchestra.run.smithi178.stdout:Setting orchestrator backend to cephadm... 2024-02-14T17:36:56.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:55 smithi178 ceph-mon[30321]: from='client.14122 v1:172.21.15.178:0/2307448660' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-14T17:36:56.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:55 smithi178 ceph-mon[30321]: from='client.14122 v1:172.21.15.178:0/2307448660' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-14T17:36:56.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:55 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:36:56.890 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.096+0000 7f618e380700 1 Processor -- start 2024-02-14T17:36:56.890 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.096+0000 7f618e380700 1 -- start start 2024-02-14T17:36:56.890 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.096+0000 7f618e380700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6188108e20 con 0x7f61881049f0 2024-02-14T17:36:56.890 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.096+0000 7f6187fff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f61881049f0 0x7f6188104e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39818/0 (socket says 172.21.15.178:39818) 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.097+0000 7f6187fff700 1 -- 172.21.15.178:0/1358167567 learned_addr learned my addr 172.21.15.178:0/1358167567 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.097+0000 7f6186ffd700 1 -- 172.21.15.178:0/1358167567 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255829553 0 0) 0x7f6188108e20 con 0x7f61881049f0 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.097+0000 7f6186ffd700 1 -- 172.21.15.178:0/1358167567 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61700035b0 con 0x7f61881049f0 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.097+0000 7f6186ffd700 1 -- 172.21.15.178:0/1358167567 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1213237035 0 0) 0x7f61700035b0 con 0x7f61881049f0 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.097+0000 7f6186ffd700 1 -- 172.21.15.178:0/1358167567 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6188109fc0 con 0x7f61881049f0 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.097+0000 7f6186ffd700 1 -- 172.21.15.178:0/1358167567 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f6178001580 con 0x7f61881049f0 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.097+0000 7f6186ffd700 1 -- 172.21.15.178:0/1358167567 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f6178001ce0 con 0x7f61881049f0 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.097+0000 7f6186ffd700 1 -- 172.21.15.178:0/1358167567 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f6178005960 con 0x7f61881049f0 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.098+0000 7f618e380700 1 -- 172.21.15.178:0/1358167567 >> v1:172.21.15.178:6789/0 conn(0x7f61881049f0 legacy=0x7f6188104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.098+0000 7f618e380700 1 -- 172.21.15.178:0/1358167567 shutdown_connections 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.098+0000 7f618e380700 1 -- 172.21.15.178:0/1358167567 >> 172.21.15.178:0/1358167567 conn(0x7f61880fff70 msgr2=0x7f61881023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.098+0000 7f618e380700 1 -- 172.21.15.178:0/1358167567 shutdown_connections 2024-02-14T17:36:56.891 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.098+0000 7f618e380700 1 -- 172.21.15.178:0/1358167567 wait complete. 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.098+0000 7f618e380700 1 Processor -- start 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.098+0000 7f618e380700 1 -- start start 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f618e380700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6188199bc0 con 0x7f61881049f0 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f6187fff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f61881049f0 0x7f61881994b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39824/0 (socket says 172.21.15.178:39824) 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f6187fff700 1 -- 172.21.15.178:0/1173962167 learned_addr learned my addr 172.21.15.178:0/1173962167 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342689445 0 0) 0x7f6188199bc0 con 0x7f61881049f0 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61700054f0 con 0x7f61881049f0 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2162823783 0 0) 0x7f61700054f0 con 0x7f61881049f0 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6188199bc0 con 0x7f61881049f0 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f6178005940 con 0x7f61881049f0 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3644544279 0 0) 0x7f6188199bc0 con 0x7f61881049f0 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6188199d50 con 0x7f61881049f0 2024-02-14T17:36:56.892 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f618e380700 1 -- 172.21.15.178:0/1173962167 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618819a000 con 0x7f61881049f0 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.099+0000 7f618e380700 1 -- 172.21.15.178:0/1173962167 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f618819db50 con 0x7f61881049f0 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.100+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f61780011c0 con 0x7f61881049f0 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.100+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f6178005f90 con 0x7f61881049f0 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.100+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 268748626 0 0) 0x7f6178011450 con 0x7f61881049f0 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.100+0000 7f618e380700 1 -- 172.21.15.178:0/1173962167 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6174005120 con 0x7f61881049f0 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.101+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7f6178047100 con 0x7f61881049f0 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.105+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f6178012cb0 con 0x7f61881049f0 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.312+0000 7f618e380700 1 -- 172.21.15.178:0/1173962167 --> v1:172.21.15.178:6800/366374299 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f6174000b90 con 0x7f617003a220 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.317+0000 7f61857fa700 1 -- 172.21.15.178:0/1173962167 <== mgr.14118 v1:172.21.15.178:6800/366374299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f6174000b90 con 0x7f617003a220 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.319+0000 7f618e380700 1 -- 172.21.15.178:0/1173962167 >> v1:172.21.15.178:6800/366374299 conn(0x7f617003a220 legacy=0x7f617003c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.319+0000 7f618e380700 1 -- 172.21.15.178:0/1173962167 >> v1:172.21.15.178:6789/0 conn(0x7f61881049f0 legacy=0x7f61881994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.319+0000 7f618e380700 1 -- 172.21.15.178:0/1173962167 shutdown_connections 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.319+0000 7f618e380700 1 -- 172.21.15.178:0/1173962167 >> 172.21.15.178:0/1173962167 conn(0x7f61880fff70 msgr2=0x7f61880788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.319+0000 7f618e380700 1 -- 172.21.15.178:0/1173962167 shutdown_connections 2024-02-14T17:36:56.893 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:56.319+0000 7f618e380700 1 -- 172.21.15.178:0/1173962167 wait complete. 2024-02-14T17:36:57.228 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:56 smithi178 ceph-mon[30321]: [14/Feb/2024:17:36:55] ENGINE Bus STARTING 2024-02-14T17:36:57.228 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:56 smithi178 ceph-mon[30321]: [14/Feb/2024:17:36:55] ENGINE Serving on http://172.21.15.178:8765 2024-02-14T17:36:57.228 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:56 smithi178 ceph-mon[30321]: [14/Feb/2024:17:36:55] ENGINE Serving on https://172.21.15.178:7150 2024-02-14T17:36:57.228 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:56 smithi178 ceph-mon[30321]: [14/Feb/2024:17:36:55] ENGINE Bus STARTED 2024-02-14T17:36:57.228 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:56 smithi178 ceph-mon[30321]: mgrmap e8: y(active, since 2s) 2024-02-14T17:36:57.229 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:56 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:36:57.229 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:56 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:36:58.107 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:57 smithi178 ceph-mon[30321]: from='client.14130 v1:172.21.15.178:0/1173962167' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:36:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.327+0000 7f10b44fc700 1 Processor -- start 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.327+0000 7f10b44fc700 1 -- start start 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.327+0000 7f10b44fc700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f10ac108e30 con 0x7f10ac104a00 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.328+0000 7f10b2298700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f10ac104a00 0x7f10ac104e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39826/0 (socket says 172.21.15.178:39826) 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.328+0000 7f10b2298700 1 -- 172.21.15.178:0/1545765004 learned_addr learned my addr 172.21.15.178:0/1545765004 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.328+0000 7f10b1296700 1 -- 172.21.15.178:0/1545765004 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286336753 0 0) 0x7f10ac108e30 con 0x7f10ac104a00 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.328+0000 7f10b1296700 1 -- 172.21.15.178:0/1545765004 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10980035b0 con 0x7f10ac104a00 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.328+0000 7f10b1296700 1 -- 172.21.15.178:0/1545765004 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 763635951 0 0) 0x7f10980035b0 con 0x7f10ac104a00 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.328+0000 7f10b1296700 1 -- 172.21.15.178:0/1545765004 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac109fd0 con 0x7f10ac104a00 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.328+0000 7f10b1296700 1 -- 172.21.15.178:0/1545765004 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f109c001580 con 0x7f10ac104a00 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.328+0000 7f10b1296700 1 -- 172.21.15.178:0/1545765004 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f109c001ce0 con 0x7f10ac104a00 2024-02-14T17:36:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.328+0000 7f10b1296700 1 -- 172.21.15.178:0/1545765004 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f109c005960 con 0x7f10ac104a00 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.329+0000 7f10b44fc700 1 -- 172.21.15.178:0/1545765004 >> v1:172.21.15.178:6789/0 conn(0x7f10ac104a00 legacy=0x7f10ac104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.329+0000 7f10b44fc700 1 -- 172.21.15.178:0/1545765004 shutdown_connections 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.329+0000 7f10b44fc700 1 -- 172.21.15.178:0/1545765004 >> 172.21.15.178:0/1545765004 conn(0x7f10ac0fff80 msgr2=0x7f10ac1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.329+0000 7f10b44fc700 1 -- 172.21.15.178:0/1545765004 shutdown_connections 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.329+0000 7f10b44fc700 1 -- 172.21.15.178:0/1545765004 wait complete. 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10b44fc700 1 Processor -- start 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10b44fc700 1 -- start start 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10b44fc700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f10ac19bd30 con 0x7f10ac104a00 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10b2298700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f10ac104a00 0x7f10ac19b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39840/0 (socket says 172.21.15.178:39840) 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10b2298700 1 -- 172.21.15.178:0/706894932 learned_addr learned my addr 172.21.15.178:0/706894932 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000164371 0 0) 0x7f10ac19bd30 con 0x7f10ac104a00 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10980054f0 con 0x7f10ac104a00 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1752793557 0 0) 0x7f10980054f0 con 0x7f10ac104a00 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10ac19bd30 con 0x7f10ac104a00 2024-02-14T17:36:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f109c005940 con 0x7f10ac104a00 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1459627841 0 0) 0x7f10ac19bd30 con 0x7f10ac104a00 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac19bec0 con 0x7f10ac104a00 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10b44fc700 1 -- 172.21.15.178:0/706894932 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac19c170 con 0x7f10ac104a00 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.330+0000 7f10b44fc700 1 -- 172.21.15.178:0/706894932 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac19fcc0 con 0x7f10ac104a00 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.331+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f109c001b60 con 0x7f10ac104a00 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.331+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f109c0061c0 con 0x7f10ac104a00 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.331+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 268748626 0 0) 0x7f109c011680 con 0x7f10ac104a00 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.331+0000 7f10b44fc700 1 -- 172.21.15.178:0/706894932 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1090005150 con 0x7f10ac104a00 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.331+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7f109c047320 con 0x7f10ac104a00 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.336+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f109c012ee0 con 0x7f10ac104a00 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.531+0000 7f10b44fc700 1 -- 172.21.15.178:0/706894932 --> v1:172.21.15.178:6800/366374299 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f1090000bc0 con 0x7f109803a120 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.531+0000 7f10a37fe700 1 -- 172.21.15.178:0/706894932 <== mgr.14118 v1:172.21.15.178:6800/366374299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (unknown 0 0 2070689548) 0x7f1090000bc0 con 0x7f109803a120 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.534+0000 7f10b44fc700 1 -- 172.21.15.178:0/706894932 >> v1:172.21.15.178:6800/366374299 conn(0x7f109803a120 legacy=0x7f109803c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:58.116 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.534+0000 7f10b44fc700 1 -- 172.21.15.178:0/706894932 >> v1:172.21.15.178:6789/0 conn(0x7f10ac104a00 legacy=0x7f10ac19b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:58.117 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.534+0000 7f10b44fc700 1 -- 172.21.15.178:0/706894932 shutdown_connections 2024-02-14T17:36:58.117 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.534+0000 7f10b44fc700 1 -- 172.21.15.178:0/706894932 >> 172.21.15.178:0/706894932 conn(0x7f10ac0fff80 msgr2=0x7f10ac1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:58.117 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.534+0000 7f10b44fc700 1 -- 172.21.15.178:0/706894932 shutdown_connections 2024-02-14T17:36:58.117 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:57.534+0000 7f10b44fc700 1 -- 172.21.15.178:0/706894932 wait complete. 2024-02-14T17:36:58.117 INFO:teuthology.orchestra.run.smithi178.stdout:Generating ssh key... 2024-02-14T17:36:59.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:58 smithi178 ceph-mon[30321]: from='client.14132 v1:172.21.15.178:0/706894932' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:36:59.480 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: Generating public/private rsa key pair. 2024-02-14T17:36:59.480 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: Your identification has been saved in /tmp/tmpl_ev8thf/key. 2024-02-14T17:36:59.480 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: Your public key has been saved in /tmp/tmpl_ev8thf/key.pub. 2024-02-14T17:36:59.480 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: The key fingerprint is: 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: SHA256:SWEYUorqf56ZM7wbOPhSelRPyqEe6tcVoPmYCImH79Y ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: The key's randomart image is: 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: +---[RSA 3072]----+ 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: | ..ooo | 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: | . +.. . | 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: |.o. + . . | 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: |=..o o + . | 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: |oo. B = S | 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: |..oB.+ o | 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: | +*++.. | 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: | +=+E*= | 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: |.o+ooB= | 2024-02-14T17:36:59.481 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: +----[SHA256]-----+ 2024-02-14T17:36:59.762 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.549+0000 7f2039281700 1 Processor -- start 2024-02-14T17:36:59.762 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.549+0000 7f2039281700 1 -- start start 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.549+0000 7f2039281700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f20340796e0 con 0x7f20341043f0 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.549+0000 7f2032ffd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f20341043f0 0x7f2034104810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39850/0 (socket says 172.21.15.178:39850) 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.549+0000 7f2032ffd700 1 -- 172.21.15.178:0/1176813484 learned_addr learned my addr 172.21.15.178:0/1176813484 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.549+0000 7f2031ffb700 1 -- 172.21.15.178:0/1176813484 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888646850 0 0) 0x7f20340796e0 con 0x7f20341043f0 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.549+0000 7f2031ffb700 1 -- 172.21.15.178:0/1176813484 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f201c0035b0 con 0x7f20341043f0 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.549+0000 7f2031ffb700 1 -- 172.21.15.178:0/1176813484 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 130465723 0 0) 0x7f201c0035b0 con 0x7f20341043f0 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.549+0000 7f2031ffb700 1 -- 172.21.15.178:0/1176813484 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2034105f30 con 0x7f20341043f0 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.549+0000 7f2031ffb700 1 -- 172.21.15.178:0/1176813484 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2024001580 con 0x7f20341043f0 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.550+0000 7f2031ffb700 1 -- 172.21.15.178:0/1176813484 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f2024001ce0 con 0x7f20341043f0 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.550+0000 7f2031ffb700 1 -- 172.21.15.178:0/1176813484 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2024005960 con 0x7f20341043f0 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.550+0000 7f2039281700 1 -- 172.21.15.178:0/1176813484 >> v1:172.21.15.178:6789/0 conn(0x7f20341043f0 legacy=0x7f2034104810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.550+0000 7f2039281700 1 -- 172.21.15.178:0/1176813484 shutdown_connections 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.550+0000 7f2039281700 1 -- 172.21.15.178:0/1176813484 >> 172.21.15.178:0/1176813484 conn(0x7f20340fffd0 msgr2=0x7f20341023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.550+0000 7f2039281700 1 -- 172.21.15.178:0/1176813484 shutdown_connections 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.550+0000 7f2039281700 1 -- 172.21.15.178:0/1176813484 wait complete. 2024-02-14T17:36:59.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.551+0000 7f2039281700 1 Processor -- start 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.551+0000 7f2039281700 1 -- start start 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.551+0000 7f2039281700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f203419bd40 con 0x7f20341043f0 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.551+0000 7f2032ffd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f20341043f0 0x7f203419b630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39854/0 (socket says 172.21.15.178:39854) 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.551+0000 7f2032ffd700 1 -- 172.21.15.178:0/494069278 learned_addr learned my addr 172.21.15.178:0/494069278 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.551+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817041886 0 0) 0x7f203419bd40 con 0x7f20341043f0 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.551+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f201c0054f0 con 0x7f20341043f0 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.552+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3077611449 0 0) 0x7f201c0054f0 con 0x7f20341043f0 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.552+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f203419bd40 con 0x7f20341043f0 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.552+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2024004870 con 0x7f20341043f0 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.552+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3646019377 0 0) 0x7f203419bd40 con 0x7f20341043f0 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.552+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203419bed0 con 0x7f20341043f0 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.552+0000 7f2039281700 1 -- 172.21.15.178:0/494069278 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203419c180 con 0x7f20341043f0 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.552+0000 7f2039281700 1 -- 172.21.15.178:0/494069278 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f203419fcd0 con 0x7f20341043f0 2024-02-14T17:36:59.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.552+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f2024004a40 con 0x7f20341043f0 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.552+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2024006110 con 0x7f20341043f0 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.553+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 268748626 0 0) 0x7f20240063e0 con 0x7f20341043f0 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.553+0000 7f2039281700 1 -- 172.21.15.178:0/494069278 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2020005150 con 0x7f20341043f0 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.553+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7f20240472d0 con 0x7f20341043f0 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.558+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f2024006c00 con 0x7f20341043f0 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:58.752+0000 7f2039281700 1 -- 172.21.15.178:0/494069278 --> v1:172.21.15.178:6800/366374299 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f2020000bc0 con 0x7f201c03a120 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:59.176+0000 7f201bfff700 1 -- 172.21.15.178:0/494069278 <== mgr.14118 v1:172.21.15.178:6800/366374299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f2020000bc0 con 0x7f201c03a120 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:59.179+0000 7f2039281700 1 -- 172.21.15.178:0/494069278 >> v1:172.21.15.178:6800/366374299 conn(0x7f201c03a120 legacy=0x7f201c03c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:59.179+0000 7f2039281700 1 -- 172.21.15.178:0/494069278 >> v1:172.21.15.178:6789/0 conn(0x7f20341043f0 legacy=0x7f203419b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:59.179+0000 7f2039281700 1 -- 172.21.15.178:0/494069278 shutdown_connections 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:59.179+0000 7f2039281700 1 -- 172.21.15.178:0/494069278 >> 172.21.15.178:0/494069278 conn(0x7f20340fffd0 msgr2=0x7f20341023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:59.179+0000 7f2039281700 1 -- 172.21.15.178:0/494069278 shutdown_connections 2024-02-14T17:36:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:36:59.179+0000 7f2039281700 1 -- 172.21.15.178:0/494069278 wait complete. 2024-02-14T17:37:00.034 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-mon[30321]: from='client.14134 v1:172.21.15.178:0/494069278' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:37:00.034 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-mon[30321]: Generating ssh key... 2024-02-14T17:37:00.034 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:37:00.034 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:36:59 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:37:01.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKfRzz3lWgq+vxvk13JzmeQ3dAA7gF6mXX3bwTUSY35G+bcMkUV+HSAnc6YTF+UEVLkyxL48zbY2uryA25S8OngYLOsg7YvgdzK7mBjLPvNB8pLcvrdN9JLxFJf0NFxc70WWQlvpSGBeCmmP7wZ2gkSwx+z9KcAE4i9svOHsQXZ55A/g0KBkuABZBI9htEs4UzIZ584eXc1Q4ufnmcKM3WD/byY4Zr62orMAr7mOWP5gNNzM4JPfhPuWYJg7bNDkJarUexlOozlxgT7bYUW1jOb/S0ZnyLizBtd2Oo8vuF1NxCpyKnNamwspw7TTcJAud2TleNvGIvI+zbb+eRaNymmZ0r1ZdHlc797i+0AEosD11D1ljtqPH7q7Pi67/s7lvPoJO7ncLX3WcOdBj/OnFI/vbeLFGqMd9Ipo6XALz0B3EFrkfdTbRTGZUiK2uz2+eNtEsMhmk3sd3pietobWqre4DEleJPnplJEtQ+pvmDKKNAzzbl/QLnbDzhkp0oH5c= ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:37:01.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.278+0000 7f5422969700 1 Processor -- start 2024-02-14T17:37:01.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.278+0000 7f5422969700 1 -- start start 2024-02-14T17:37:01.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.278+0000 7f5422969700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f541c108e30 con 0x7f541c104a00 2024-02-14T17:37:01.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.279+0000 7f541bfff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f541c104a00 0x7f541c104e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39868/0 (socket says 172.21.15.178:39868) 2024-02-14T17:37:01.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.279+0000 7f541bfff700 1 -- 172.21.15.178:0/2200646249 learned_addr learned my addr 172.21.15.178:0/2200646249 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:01.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.279+0000 7f541affd700 1 -- 172.21.15.178:0/2200646249 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208951750 0 0) 0x7f541c108e30 con 0x7f541c104a00 2024-02-14T17:37:01.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.279+0000 7f541affd700 1 -- 172.21.15.178:0/2200646249 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54040035b0 con 0x7f541c104a00 2024-02-14T17:37:01.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.279+0000 7f541affd700 1 -- 172.21.15.178:0/2200646249 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3211100917 0 0) 0x7f54040035b0 con 0x7f541c104a00 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.279+0000 7f541affd700 1 -- 172.21.15.178:0/2200646249 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c109fd0 con 0x7f541c104a00 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.279+0000 7f541affd700 1 -- 172.21.15.178:0/2200646249 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f540c001580 con 0x7f541c104a00 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.279+0000 7f541affd700 1 -- 172.21.15.178:0/2200646249 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f540c001ce0 con 0x7f541c104a00 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.279+0000 7f541affd700 1 -- 172.21.15.178:0/2200646249 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f540c005960 con 0x7f541c104a00 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.280+0000 7f5422969700 1 -- 172.21.15.178:0/2200646249 >> v1:172.21.15.178:6789/0 conn(0x7f541c104a00 legacy=0x7f541c104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.280+0000 7f5422969700 1 -- 172.21.15.178:0/2200646249 shutdown_connections 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.280+0000 7f5422969700 1 -- 172.21.15.178:0/2200646249 >> 172.21.15.178:0/2200646249 conn(0x7f541c0fff80 msgr2=0x7f541c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.280+0000 7f5422969700 1 -- 172.21.15.178:0/2200646249 shutdown_connections 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.280+0000 7f5422969700 1 -- 172.21.15.178:0/2200646249 wait complete. 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.280+0000 7f5422969700 1 Processor -- start 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.281+0000 7f5422969700 1 -- start start 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.281+0000 7f5422969700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f541c078a90 con 0x7f541c104a00 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.281+0000 7f541bfff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f541c104a00 0x7f541c19c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39882/0 (socket says 172.21.15.178:39882) 2024-02-14T17:37:01.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.281+0000 7f541bfff700 1 -- 172.21.15.178:0/1505982566 learned_addr learned my addr 172.21.15.178:0/1505982566 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.281+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13788784 0 0) 0x7f541c078a90 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.281+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54040054f0 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.281+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 335343468 0 0) 0x7f54040054f0 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.281+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f541c078a90 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.281+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f540c005940 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.282+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1262774527 0 0) 0x7f541c078a90 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.282+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c078c20 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.282+0000 7f5422969700 1 -- 172.21.15.178:0/1505982566 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c078ed0 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.282+0000 7f5422969700 1 -- 172.21.15.178:0/1505982566 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c19dab0 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.282+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f540c001080 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.282+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f540c005fd0 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.283+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 268748626 0 0) 0x7f540c011570 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.283+0000 7f5422969700 1 -- 172.21.15.178:0/1505982566 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5408005150 con 0x7f541c104a00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.283+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7f540c0472c0 con 0x7f541c104a00 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.289+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f540c012e30 con 0x7f541c104a00 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.483+0000 7f5422969700 1 -- 172.21.15.178:0/1505982566 --> v1:172.21.15.178:6800/366374299 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f5408000bc0 con 0x7f540403a170 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.484+0000 7f54197fa700 1 -- 172.21.15.178:0/1505982566 <== mgr.14118 v1:172.21.15.178:6800/366374299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (unknown 0 0 4252948047) 0x7f5408000bc0 con 0x7f540403a170 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.486+0000 7f5422969700 1 -- 172.21.15.178:0/1505982566 >> v1:172.21.15.178:6800/366374299 conn(0x7f540403a170 legacy=0x7f540403c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.486+0000 7f5422969700 1 -- 172.21.15.178:0/1505982566 >> v1:172.21.15.178:6789/0 conn(0x7f541c104a00 legacy=0x7f541c19c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.486+0000 7f5422969700 1 -- 172.21.15.178:0/1505982566 shutdown_connections 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.486+0000 7f5422969700 1 -- 172.21.15.178:0/1505982566 >> 172.21.15.178:0/1505982566 conn(0x7f541c0fff80 msgr2=0x7f541c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.486+0000 7f5422969700 1 -- 172.21.15.178:0/1505982566 shutdown_connections 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:00.486+0000 7f5422969700 1 -- 172.21.15.178:0/1505982566 wait complete. 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:Adding key to root@localhost authorized_keys... 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi178.stdout:Adding host smithi178... 2024-02-14T17:37:01.424 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:01 smithi178 ceph-mon[30321]: from='client.14136 v1:172.21.15.178:0/1505982566' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:37:02.255 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:02 smithi178 ceph-mon[30321]: from='client.14138 v1:172.21.15.178:0/1943923707' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "addr": "172.21.15.178", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:37:03.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:03 smithi178 ceph-mon[30321]: Deploying cephadm binary to smithi178 2024-02-14T17:37:06.453 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout Added host 'smithi178' with addr '172.21.15.178' 2024-02-14T17:37:06.453 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.526+0000 7fdddfe14700 1 Processor -- start 2024-02-14T17:37:06.453 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.526+0000 7fdddfe14700 1 -- start start 2024-02-14T17:37:06.453 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.526+0000 7fdddfe14700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fddd8109f40 con 0x7fddd8105b10 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.526+0000 7fddddbb0700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fddd8105b10 0x7fddd8105f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39886/0 (socket says 172.21.15.178:39886) 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.526+0000 7fddddbb0700 1 -- 172.21.15.178:0/3651960807 learned_addr learned my addr 172.21.15.178:0/3651960807 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.526+0000 7fdddcbae700 1 -- 172.21.15.178:0/3651960807 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689849000 0 0) 0x7fddd8109f40 con 0x7fddd8105b10 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.526+0000 7fdddcbae700 1 -- 172.21.15.178:0/3651960807 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddc40035b0 con 0x7fddd8105b10 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.527+0000 7fdddcbae700 1 -- 172.21.15.178:0/3651960807 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1682354008 0 0) 0x7fddc40035b0 con 0x7fddd8105b10 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.527+0000 7fdddcbae700 1 -- 172.21.15.178:0/3651960807 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd810b0e0 con 0x7fddd8105b10 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.527+0000 7fdddcbae700 1 -- 172.21.15.178:0/3651960807 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fddd4001580 con 0x7fddd8105b10 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.527+0000 7fdddcbae700 1 -- 172.21.15.178:0/3651960807 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7fddd4001ce0 con 0x7fddd8105b10 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.527+0000 7fdddcbae700 1 -- 172.21.15.178:0/3651960807 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fddd4005960 con 0x7fddd8105b10 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.527+0000 7fdddfe14700 1 -- 172.21.15.178:0/3651960807 >> v1:172.21.15.178:6789/0 conn(0x7fddd8105b10 legacy=0x7fddd8105f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.527+0000 7fdddfe14700 1 -- 172.21.15.178:0/3651960807 shutdown_connections 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.527+0000 7fdddfe14700 1 -- 172.21.15.178:0/3651960807 >> 172.21.15.178:0/3651960807 conn(0x7fddd81010b0 msgr2=0x7fddd81034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.527+0000 7fdddfe14700 1 -- 172.21.15.178:0/3651960807 shutdown_connections 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.527+0000 7fdddfe14700 1 -- 172.21.15.178:0/3651960807 wait complete. 2024-02-14T17:37:06.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.528+0000 7fdddfe14700 1 Processor -- start 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.528+0000 7fdddfe14700 1 -- start start 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.528+0000 7fdddfe14700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fddd8199b20 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.528+0000 7fddddbb0700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fddd8105b10 0x7fddd8199410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:39898/0 (socket says 172.21.15.178:39898) 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.528+0000 7fddddbb0700 1 -- 172.21.15.178:0/1943923707 learned_addr learned my addr 172.21.15.178:0/1943923707 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.528+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473653643 0 0) 0x7fddd8199b20 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.528+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddc40054f0 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.529+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4106533121 0 0) 0x7fddc40054f0 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.529+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddd8199b20 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.529+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fddd4005940 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.529+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1681280932 0 0) 0x7fddd8199b20 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.529+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd8199cb0 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.529+0000 7fdddfe14700 1 -- 172.21.15.178:0/1943923707 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd8199f60 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.529+0000 7fdddfe14700 1 -- 172.21.15.178:0/1943923707 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd819dab0 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.529+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7fddd4001b20 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.529+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fddd40060b0 con 0x7fddd8105b10 2024-02-14T17:37:06.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.530+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 268748626 0 0) 0x7fddd4011650 con 0x7fddd8105b10 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.530+0000 7fdddfe14700 1 -- 172.21.15.178:0/1943923707 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddbc005150 con 0x7fddd8105b10 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.530+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7fddd4047540 con 0x7fddd8105b10 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.535+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fddd4012ec0 con 0x7fddd8105b10 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:01.729+0000 7fdddfe14700 1 -- 172.21.15.178:0/1943923707 --> v1:172.21.15.178:6800/366374299 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi178", "addr": "172.21.15.178", "target": ["mon-mgr", ""]}) v1 -- 0x7fddbc000bc0 con 0x7fddc403a120 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:05.896+0000 7fddceffd700 1 -- 172.21.15.178:0/1943923707 <== mgr.14118 v1:172.21.15.178:6800/366374299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 1801797780) 0x7fddbc000bc0 con 0x7fddc403a120 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:05.899+0000 7fdddfe14700 1 -- 172.21.15.178:0/1943923707 >> v1:172.21.15.178:6800/366374299 conn(0x7fddc403a120 legacy=0x7fddc403c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:05.899+0000 7fdddfe14700 1 -- 172.21.15.178:0/1943923707 >> v1:172.21.15.178:6789/0 conn(0x7fddd8105b10 legacy=0x7fddd8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:05.899+0000 7fdddfe14700 1 -- 172.21.15.178:0/1943923707 shutdown_connections 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:05.899+0000 7fdddfe14700 1 -- 172.21.15.178:0/1943923707 >> 172.21.15.178:0/1943923707 conn(0x7fddd81010b0 msgr2=0x7fddd81034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:05.899+0000 7fdddfe14700 1 -- 172.21.15.178:0/1943923707 shutdown_connections 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:05.899+0000 7fdddfe14700 1 -- 172.21.15.178:0/1943923707 wait complete. 2024-02-14T17:37:06.456 INFO:teuthology.orchestra.run.smithi178.stdout:Deploying unmanaged mon service... 2024-02-14T17:37:07.163 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:06 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:37:07.163 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:06 smithi178 ceph-mon[30321]: Added host smithi178 2024-02-14T17:37:07.163 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:06 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:37:07.787 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-14T17:37:07.787 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.944+0000 7f32e83cf700 1 Processor -- start 2024-02-14T17:37:07.787 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.944+0000 7f32e83cf700 1 -- start start 2024-02-14T17:37:07.787 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.944+0000 7f32e83cf700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f32e0109f40 con 0x7f32e0105b10 2024-02-14T17:37:07.787 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.944+0000 7f32e616b700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f32e0105b10 0x7f32e0105f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56802/0 (socket says 172.21.15.178:56802) 2024-02-14T17:37:07.787 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.944+0000 7f32e616b700 1 -- 172.21.15.178:0/3019254172 learned_addr learned my addr 172.21.15.178:0/3019254172 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:07.787 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.945+0000 7f32e5169700 1 -- 172.21.15.178:0/3019254172 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884758633 0 0) 0x7f32e0109f40 con 0x7f32e0105b10 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.945+0000 7f32e5169700 1 -- 172.21.15.178:0/3019254172 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32cc0035b0 con 0x7f32e0105b10 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.945+0000 7f32e5169700 1 -- 172.21.15.178:0/3019254172 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2138112103 0 0) 0x7f32cc0035b0 con 0x7f32e0105b10 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.945+0000 7f32e5169700 1 -- 172.21.15.178:0/3019254172 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e010b0e0 con 0x7f32e0105b10 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.945+0000 7f32e5169700 1 -- 172.21.15.178:0/3019254172 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f32dc001580 con 0x7f32e0105b10 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.945+0000 7f32e5169700 1 -- 172.21.15.178:0/3019254172 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f32dc001ce0 con 0x7f32e0105b10 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.945+0000 7f32e5169700 1 -- 172.21.15.178:0/3019254172 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f32dc005960 con 0x7f32e0105b10 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.945+0000 7f32e83cf700 1 -- 172.21.15.178:0/3019254172 >> v1:172.21.15.178:6789/0 conn(0x7f32e0105b10 legacy=0x7f32e0105f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.946+0000 7f32e83cf700 1 -- 172.21.15.178:0/3019254172 shutdown_connections 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.946+0000 7f32e83cf700 1 -- 172.21.15.178:0/3019254172 >> 172.21.15.178:0/3019254172 conn(0x7f32e01010b0 msgr2=0x7f32e01034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.946+0000 7f32e83cf700 1 -- 172.21.15.178:0/3019254172 shutdown_connections 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.946+0000 7f32e83cf700 1 -- 172.21.15.178:0/3019254172 wait complete. 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.946+0000 7f32e83cf700 1 Processor -- start 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.946+0000 7f32e83cf700 1 -- start start 2024-02-14T17:37:07.788 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.946+0000 7f32e83cf700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f32e0199b50 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.946+0000 7f32e616b700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f32e0105b10 0x7f32e0199440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56816/0 (socket says 172.21.15.178:56816) 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.946+0000 7f32e616b700 1 -- 172.21.15.178:0/2727514348 learned_addr learned my addr 172.21.15.178:0/2727514348 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.947+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656574747 0 0) 0x7f32e0199b50 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.947+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32cc0054f0 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.947+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3162283018 0 0) 0x7f32cc0054f0 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.947+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f32e0199b50 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.947+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f32dc005940 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.947+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 52862771 0 0) 0x7f32e0199b50 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.947+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0199ce0 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.947+0000 7f32e83cf700 1 -- 172.21.15.178:0/2727514348 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e0199f90 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.947+0000 7f32e83cf700 1 -- 172.21.15.178:0/2727514348 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e019dae0 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.947+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7f32dc001080 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.947+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f32dc005ff0 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.948+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 268748626 0 0) 0x7f32dc0114d0 con 0x7f32e0105b10 2024-02-14T17:37:07.789 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.948+0000 7f32e83cf700 1 -- 172.21.15.178:0/2727514348 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c4005150 con 0x7f32e0105b10 2024-02-14T17:37:07.790 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.948+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7f32dc047370 con 0x7f32e0105b10 2024-02-14T17:37:07.790 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:06.953+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f32dc012d30 con 0x7f32e0105b10 2024-02-14T17:37:07.790 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:07.161+0000 7f32e83cf700 1 -- 172.21.15.178:0/2727514348 --> v1:172.21.15.178:6800/366374299 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f32c4000bc0 con 0x7f32cc03a1c0 2024-02-14T17:37:07.790 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:07.166+0000 7f32d77fe700 1 -- 172.21.15.178:0/2727514348 <== mgr.14118 v1:172.21.15.178:6800/366374299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7f32c4000bc0 con 0x7f32cc03a1c0 2024-02-14T17:37:07.790 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:07.168+0000 7f32e83cf700 1 -- 172.21.15.178:0/2727514348 >> v1:172.21.15.178:6800/366374299 conn(0x7f32cc03a1c0 legacy=0x7f32cc03c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:07.790 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:07.168+0000 7f32e83cf700 1 -- 172.21.15.178:0/2727514348 >> v1:172.21.15.178:6789/0 conn(0x7f32e0105b10 legacy=0x7f32e0199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:07.790 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:07.168+0000 7f32e83cf700 1 -- 172.21.15.178:0/2727514348 shutdown_connections 2024-02-14T17:37:07.790 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:07.168+0000 7f32e83cf700 1 -- 172.21.15.178:0/2727514348 >> 172.21.15.178:0/2727514348 conn(0x7f32e01010b0 msgr2=0x7f32e01034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:07.790 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:07.168+0000 7f32e83cf700 1 -- 172.21.15.178:0/2727514348 shutdown_connections 2024-02-14T17:37:07.790 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:07.168+0000 7f32e83cf700 1 -- 172.21.15.178:0/2727514348 wait complete. 2024-02-14T17:37:07.790 INFO:teuthology.orchestra.run.smithi178.stdout:Deploying unmanaged mgr service... 2024-02-14T17:37:08.509 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:08 smithi178 ceph-mon[30321]: from='client.14140 v1:172.21.15.178:0/2727514348' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:37:08.509 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:08 smithi178 ceph-mon[30321]: Saving service mon spec with placement count:5 2024-02-14T17:37:08.510 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:08 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:37:09.828 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:09 smithi178 ceph-mon[30321]: from='client.14142 v1:172.21.15.178:0/3699827625' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:37:09.828 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:09 smithi178 ceph-mon[30321]: Saving service mgr spec with placement count:2 2024-02-14T17:37:09.828 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:09 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:37:09.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-14T17:37:09.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.309+0000 7fe5adb11700 1 Processor -- start 2024-02-14T17:37:09.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.309+0000 7fe5adb11700 1 -- start start 2024-02-14T17:37:09.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.309+0000 7fe5adb11700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe5a8109f40 con 0x7fe5a8105b10 2024-02-14T17:37:09.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.309+0000 7fe5a77fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe5a8105b10 0x7fe5a8105f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56830/0 (socket says 172.21.15.178:56830) 2024-02-14T17:37:09.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.309+0000 7fe5a77fe700 1 -- 172.21.15.178:0/1437039656 learned_addr learned my addr 172.21.15.178:0/1437039656 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:09.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.309+0000 7fe5a67fc700 1 -- 172.21.15.178:0/1437039656 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851477864 0 0) 0x7fe5a8109f40 con 0x7fe5a8105b10 2024-02-14T17:37:09.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.310+0000 7fe5a67fc700 1 -- 172.21.15.178:0/1437039656 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5900035b0 con 0x7fe5a8105b10 2024-02-14T17:37:09.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.310+0000 7fe5a67fc700 1 -- 172.21.15.178:0/1437039656 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3073249069 0 0) 0x7fe5900035b0 con 0x7fe5a8105b10 2024-02-14T17:37:09.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.310+0000 7fe5a67fc700 1 -- 172.21.15.178:0/1437039656 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a810b0e0 con 0x7fe5a8105b10 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.310+0000 7fe5a67fc700 1 -- 172.21.15.178:0/1437039656 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fe598001580 con 0x7fe5a8105b10 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.310+0000 7fe5a67fc700 1 -- 172.21.15.178:0/1437039656 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7fe598001ce0 con 0x7fe5a8105b10 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.310+0000 7fe5a67fc700 1 -- 172.21.15.178:0/1437039656 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fe598005960 con 0x7fe5a8105b10 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.310+0000 7fe5adb11700 1 -- 172.21.15.178:0/1437039656 >> v1:172.21.15.178:6789/0 conn(0x7fe5a8105b10 legacy=0x7fe5a8105f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.310+0000 7fe5adb11700 1 -- 172.21.15.178:0/1437039656 shutdown_connections 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.310+0000 7fe5adb11700 1 -- 172.21.15.178:0/1437039656 >> 172.21.15.178:0/1437039656 conn(0x7fe5a81010b0 msgr2=0x7fe5a81034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.310+0000 7fe5adb11700 1 -- 172.21.15.178:0/1437039656 shutdown_connections 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.310+0000 7fe5adb11700 1 -- 172.21.15.178:0/1437039656 wait complete. 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.311+0000 7fe5adb11700 1 Processor -- start 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.311+0000 7fe5adb11700 1 -- start start 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.311+0000 7fe5adb11700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe5a8199b50 con 0x7fe5a8105b10 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.311+0000 7fe5a77fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe5a8105b10 0x7fe5a8199440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56846/0 (socket says 172.21.15.178:56846) 2024-02-14T17:37:09.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.311+0000 7fe5a77fe700 1 -- 172.21.15.178:0/3699827625 learned_addr learned my addr 172.21.15.178:0/3699827625 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.312+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448868788 0 0) 0x7fe5a8199b50 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.312+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5900054f0 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.312+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2050822749 0 0) 0x7fe5900054f0 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.312+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe5a8199b50 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.312+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fe598005940 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.312+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 650459724 0 0) 0x7fe5a8199b50 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.312+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a8199ce0 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.312+0000 7fe5adb11700 1 -- 172.21.15.178:0/3699827625 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a8199f90 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.312+0000 7fe5adb11700 1 -- 172.21.15.178:0/3699827625 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a819dae0 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.312+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 1007442722 0 0) 0x7fe598001b20 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.312+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fe5980060b0 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.313+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 268748626 0 0) 0x7fe598011570 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.313+0000 7fe5adb11700 1 -- 172.21.15.178:0/3699827625 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe594005150 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.313+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7fe598047250 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.318+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fe598012f60 con 0x7fe5a8105b10 2024-02-14T17:37:09.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.527+0000 7fe5adb11700 1 -- 172.21.15.178:0/3699827625 --> v1:172.21.15.178:6800/366374299 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe594000bc0 con 0x7fe59003a1c0 2024-02-14T17:37:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.531+0000 7fe5a4ff9700 1 -- 172.21.15.178:0/3699827625 <== mgr.14118 v1:172.21.15.178:6800/366374299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7fe594000bc0 con 0x7fe59003a1c0 2024-02-14T17:37:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.533+0000 7fe5adb11700 1 -- 172.21.15.178:0/3699827625 >> v1:172.21.15.178:6800/366374299 conn(0x7fe59003a1c0 legacy=0x7fe59003c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.534+0000 7fe5adb11700 1 -- 172.21.15.178:0/3699827625 >> v1:172.21.15.178:6789/0 conn(0x7fe5a8105b10 legacy=0x7fe5a8199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.534+0000 7fe5adb11700 1 -- 172.21.15.178:0/3699827625 shutdown_connections 2024-02-14T17:37:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.534+0000 7fe5adb11700 1 -- 172.21.15.178:0/3699827625 >> 172.21.15.178:0/3699827625 conn(0x7fe5a81010b0 msgr2=0x7fe5a81034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.534+0000 7fe5adb11700 1 -- 172.21.15.178:0/3699827625 shutdown_connections 2024-02-14T17:37:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:08.534+0000 7fe5adb11700 1 -- 172.21.15.178:0/3699827625 wait complete. 2024-02-14T17:37:10.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:10 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:37:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:10 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:37:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/753530988' entity='client.admin' 2024-02-14T17:37:11.132 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.313+0000 7f2c9b3a9700 1 Processor -- start 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.313+0000 7f2c9b3a9700 1 -- start start 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.313+0000 7f2c9b3a9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2c94079720 con 0x7f2c941065d0 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.313+0000 7f2c99145700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2c941065d0 0x7f2c941089c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56850/0 (socket says 172.21.15.178:56850) 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.313+0000 7f2c99145700 1 -- 172.21.15.178:0/4225711537 learned_addr learned my addr 172.21.15.178:0/4225711537 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.314+0000 7f2c8bfff700 1 -- 172.21.15.178:0/4225711537 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526873844 0 0) 0x7f2c94079720 con 0x7f2c941065d0 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.314+0000 7f2c8bfff700 1 -- 172.21.15.178:0/4225711537 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c800035b0 con 0x7f2c941065d0 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.314+0000 7f2c8bfff700 1 -- 172.21.15.178:0/4225711537 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2595493860 0 0) 0x7f2c800035b0 con 0x7f2c941065d0 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.314+0000 7f2c8bfff700 1 -- 172.21.15.178:0/4225711537 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9410b0f0 con 0x7f2c941065d0 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.314+0000 7f2c8bfff700 1 -- 172.21.15.178:0/4225711537 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2c90001580 con 0x7f2c941065d0 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.314+0000 7f2c8bfff700 1 -- 172.21.15.178:0/4225711537 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f2c90001d00 con 0x7f2c941065d0 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.314+0000 7f2c8bfff700 1 -- 172.21.15.178:0/4225711537 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2c90005960 con 0x7f2c941065d0 2024-02-14T17:37:11.133 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.315+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/4225711537 >> v1:172.21.15.178:6789/0 conn(0x7f2c941065d0 legacy=0x7f2c941089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.315+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/4225711537 shutdown_connections 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.315+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/4225711537 >> 172.21.15.178:0/4225711537 conn(0x7f2c940fffa0 msgr2=0x7f2c94102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.315+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/4225711537 shutdown_connections 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.315+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/4225711537 wait complete. 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.316+0000 7f2c9b3a9700 1 Processor -- start 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.316+0000 7f2c9b3a9700 1 -- start start 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.316+0000 7f2c9b3a9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2c94199b10 con 0x7f2c941065d0 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.316+0000 7f2c99145700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2c941065d0 0x7f2c94199400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56856/0 (socket says 172.21.15.178:56856) 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.316+0000 7f2c99145700 1 -- 172.21.15.178:0/753530988 learned_addr learned my addr 172.21.15.178:0/753530988 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.316+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259364600 0 0) 0x7f2c94199b10 con 0x7f2c941065d0 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.316+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c800054f0 con 0x7f2c941065d0 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.317+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3993193851 0 0) 0x7f2c800054f0 con 0x7f2c941065d0 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.317+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c94199b10 con 0x7f2c941065d0 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.317+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2c90001080 con 0x7f2c941065d0 2024-02-14T17:37:11.134 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.317+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 19728587 0 0) 0x7f2c94199b10 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.317+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c94199ca0 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.317+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/753530988 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c94199f50 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.317+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/753530988 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9419daa0 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.317+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f2c90004a60 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.317+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2c90006190 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.318+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 268748626 0 0) 0x7f2c90011670 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.318+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/753530988 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c78005150 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.318+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7f2c900472a0 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.323+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f2c90012e90 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.527+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/753530988 --> v1:172.21.15.178:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f2c78005400 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.529+0000 7f2c8a7fc700 1 -- 172.21.15.178:0/753530988 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (unknown 2684808024 0 0) 0x7f2c90014f70 con 0x7f2c941065d0 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.532+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/753530988 >> v1:172.21.15.178:6800/366374299 conn(0x7f2c8003a170 legacy=0x7f2c8003c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.532+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/753530988 >> v1:172.21.15.178:6789/0 conn(0x7f2c941065d0 legacy=0x7f2c94199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.532+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/753530988 shutdown_connections 2024-02-14T17:37:11.135 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.532+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/753530988 >> 172.21.15.178:0/753530988 conn(0x7f2c940fffa0 msgr2=0x7f2c941902b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:11.136 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.532+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/753530988 shutdown_connections 2024-02-14T17:37:11.136 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:10.532+0000 7f2c9b3a9700 1 -- 172.21.15.178:0/753530988 wait complete. 2024-02-14T17:37:12.425 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.581+0000 7feaf47c9700 1 Processor -- start 2024-02-14T17:37:12.425 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.581+0000 7feaf47c9700 1 -- start start 2024-02-14T17:37:12.425 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.582+0000 7feaf47c9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7feaec07a370 con 0x7feaec07b770 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.582+0000 7feaf2565700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7feaec07b770 0x7feaec079bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56870/0 (socket says 172.21.15.178:56870) 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.582+0000 7feaf2565700 1 -- 172.21.15.178:0/2979561307 learned_addr learned my addr 172.21.15.178:0/2979561307 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.582+0000 7feaf1563700 1 -- 172.21.15.178:0/2979561307 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143422052 0 0) 0x7feaec07a370 con 0x7feaec07b770 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.582+0000 7feaf1563700 1 -- 172.21.15.178:0/2979561307 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead80035b0 con 0x7feaec07b770 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.583+0000 7feaf1563700 1 -- 172.21.15.178:0/2979561307 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2131985270 0 0) 0x7fead80035b0 con 0x7feaec07b770 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.583+0000 7feaf1563700 1 -- 172.21.15.178:0/2979561307 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec07a500 con 0x7feaec07b770 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.583+0000 7feaf1563700 1 -- 172.21.15.178:0/2979561307 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7feadc001580 con 0x7feaec07b770 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.583+0000 7feaf1563700 1 -- 172.21.15.178:0/2979561307 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7feadc001d00 con 0x7feaec07b770 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.583+0000 7feaf1563700 1 -- 172.21.15.178:0/2979561307 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7feadc005960 con 0x7feaec07b770 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.583+0000 7feaf47c9700 1 -- 172.21.15.178:0/2979561307 >> v1:172.21.15.178:6789/0 conn(0x7feaec07b770 legacy=0x7feaec079bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.583+0000 7feaf47c9700 1 -- 172.21.15.178:0/2979561307 shutdown_connections 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.583+0000 7feaf47c9700 1 -- 172.21.15.178:0/2979561307 >> 172.21.15.178:0/2979561307 conn(0x7feaec1010b0 msgr2=0x7feaec1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.583+0000 7feaf47c9700 1 -- 172.21.15.178:0/2979561307 shutdown_connections 2024-02-14T17:37:12.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.583+0000 7feaf47c9700 1 -- 172.21.15.178:0/2979561307 wait complete. 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.584+0000 7feaf47c9700 1 Processor -- start 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.584+0000 7feaf47c9700 1 -- start start 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.584+0000 7feaf47c9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7feaec19bd90 con 0x7feaec07b770 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.584+0000 7feaf2565700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7feaec07b770 0x7feaec19b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56882/0 (socket says 172.21.15.178:56882) 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.584+0000 7feaf2565700 1 -- 172.21.15.178:0/2079718357 learned_addr learned my addr 172.21.15.178:0/2079718357 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.584+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949233762 0 0) 0x7feaec19bd90 con 0x7feaec07b770 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.584+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead80054f0 con 0x7feaec07b770 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.585+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 804509181 0 0) 0x7fead80054f0 con 0x7feaec07b770 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.585+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feaec19bd90 con 0x7feaec07b770 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.585+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7feadc001080 con 0x7feaec07b770 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.585+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3287643368 0 0) 0x7feaec19bd90 con 0x7feaec07b770 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.585+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec19bf20 con 0x7feaec07b770 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.585+0000 7feaf47c9700 1 -- 172.21.15.178:0/2079718357 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec19c1d0 con 0x7feaec07b770 2024-02-14T17:37:12.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.585+0000 7feaf47c9700 1 -- 172.21.15.178:0/2079718357 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec19fd20 con 0x7feaec07b770 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.585+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7feadc004a60 con 0x7feaec07b770 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.585+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7feadc006160 con 0x7feaec07b770 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.586+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 268748626 0 0) 0x7feadc011640 con 0x7feaec07b770 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.586+0000 7feaf47c9700 1 -- 172.21.15.178:0/2079718357 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead0005150 con 0x7feaec07b770 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.586+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7feadc047230 con 0x7feaec07b770 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.591+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7feadc012e60 con 0x7feaec07b770 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.798+0000 7feaf47c9700 1 -- 172.21.15.178:0/2079718357 --> v1:172.21.15.178:6789/0 -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fead0005980 con 0x7feaec07b770 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.800+0000 7feae37fe700 1 -- 172.21.15.178:0/2079718357 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (unknown 1496729260 0 0) 0x7feadc014f40 con 0x7feaec07b770 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.802+0000 7feaf47c9700 1 -- 172.21.15.178:0/2079718357 >> v1:172.21.15.178:6800/366374299 conn(0x7fead803a170 legacy=0x7fead803c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.802+0000 7feaf47c9700 1 -- 172.21.15.178:0/2079718357 >> v1:172.21.15.178:6789/0 conn(0x7feaec07b770 legacy=0x7feaec19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.802+0000 7feaf47c9700 1 -- 172.21.15.178:0/2079718357 shutdown_connections 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.802+0000 7feaf47c9700 1 -- 172.21.15.178:0/2079718357 >> 172.21.15.178:0/2079718357 conn(0x7feaec1010b0 msgr2=0x7feaec109f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.802+0000 7feaf47c9700 1 -- 172.21.15.178:0/2079718357 shutdown_connections 2024-02-14T17:37:12.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:11.802+0000 7feaf47c9700 1 -- 172.21.15.178:0/2079718357 wait complete. 2024-02-14T17:37:12.429 INFO:teuthology.orchestra.run.smithi178.stdout:Enabling the dashboard module... 2024-02-14T17:37:13.081 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2079718357' entity='client.admin' 2024-02-14T17:37:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2786502034' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-14T17:37:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:13 smithi178 ceph-mon[30321]: from='mgr.14118 v1:172.21.15.178:0/2030864306' entity='mgr.y' 2024-02-14T17:37:14.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ignoring --setuser ceph since I am not root 2024-02-14T17:37:14.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ignoring --setgroup ceph since I am not root 2024-02-14T17:37:14.402 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.991+0000 7f5ada575700 1 Processor -- start 2024-02-14T17:37:14.402 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.991+0000 7f5ada575700 1 -- start start 2024-02-14T17:37:14.402 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.991+0000 7f5ada575700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5ad407a370 con 0x7f5ad407b770 2024-02-14T17:37:14.402 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.991+0000 7f5ad3fff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f5ad407b770 0x7f5ad4079bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56896/0 (socket says 172.21.15.178:56896) 2024-02-14T17:37:14.403 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.991+0000 7f5ad3fff700 1 -- 172.21.15.178:0/3002214377 learned_addr learned my addr 172.21.15.178:0/3002214377 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:14.403 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.991+0000 7f5ad2ffd700 1 -- 172.21.15.178:0/3002214377 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433175806 0 0) 0x7f5ad407a370 con 0x7f5ad407b770 2024-02-14T17:37:14.403 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.991+0000 7f5ad2ffd700 1 -- 172.21.15.178:0/3002214377 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5abc0035b0 con 0x7f5ad407b770 2024-02-14T17:37:14.403 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.991+0000 7f5ad2ffd700 1 -- 172.21.15.178:0/3002214377 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 655705547 0 0) 0x7f5abc0035b0 con 0x7f5ad407b770 2024-02-14T17:37:14.403 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.991+0000 7f5ad2ffd700 1 -- 172.21.15.178:0/3002214377 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad407a500 con 0x7f5ad407b770 2024-02-14T17:37:14.403 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.991+0000 7f5ad2ffd700 1 -- 172.21.15.178:0/3002214377 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f5ac4001580 con 0x7f5ad407b770 2024-02-14T17:37:14.403 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.992+0000 7f5ad2ffd700 1 -- 172.21.15.178:0/3002214377 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f5ac4001d00 con 0x7f5ad407b770 2024-02-14T17:37:14.403 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.992+0000 7f5ad2ffd700 1 -- 172.21.15.178:0/3002214377 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f5ac4005960 con 0x7f5ad407b770 2024-02-14T17:37:14.403 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.992+0000 7f5ada575700 1 -- 172.21.15.178:0/3002214377 >> v1:172.21.15.178:6789/0 conn(0x7f5ad407b770 legacy=0x7f5ad4079bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:14.403 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.992+0000 7f5ada575700 1 -- 172.21.15.178:0/3002214377 shutdown_connections 2024-02-14T17:37:14.403 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.992+0000 7f5ada575700 1 -- 172.21.15.178:0/3002214377 >> 172.21.15.178:0/3002214377 conn(0x7f5ad4101040 msgr2=0x7f5ad41034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.992+0000 7f5ada575700 1 -- 172.21.15.178:0/3002214377 shutdown_connections 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.992+0000 7f5ada575700 1 -- 172.21.15.178:0/3002214377 wait complete. 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.993+0000 7f5ada575700 1 Processor -- start 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.993+0000 7f5ada575700 1 -- start start 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.993+0000 7f5ada575700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5ad419bd70 con 0x7f5ad407b770 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.993+0000 7f5ad3fff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f5ad407b770 0x7f5ad419b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56912/0 (socket says 172.21.15.178:56912) 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.993+0000 7f5ad3fff700 1 -- 172.21.15.178:0/2786502034 learned_addr learned my addr 172.21.15.178:0/2786502034 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.993+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905565853 0 0) 0x7f5ad419bd70 con 0x7f5ad407b770 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.993+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5abc0054f0 con 0x7f5ad407b770 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.993+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3864297710 0 0) 0x7f5abc0054f0 con 0x7f5ad407b770 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.994+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ad419bd70 con 0x7f5ad407b770 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.994+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f5ac4001080 con 0x7f5ad407b770 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.994+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 698711988 0 0) 0x7f5ad419bd70 con 0x7f5ad407b770 2024-02-14T17:37:14.404 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.994+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad419bf00 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.994+0000 7f5ada575700 1 -- 172.21.15.178:0/2786502034 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad419c1b0 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.994+0000 7f5ada575700 1 -- 172.21.15.178:0/2786502034 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad419fd00 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.994+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f5ac4004a60 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.994+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f5ac4006210 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.995+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 268748626 0 0) 0x7f5ac40117b0 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.995+0000 7f5ada575700 1 -- 172.21.15.178:0/2786502034 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac0005150 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.995+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7f5ac4047440 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:12.999+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f5ac4013020 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:13.251+0000 7f5ada575700 1 -- 172.21.15.178:0/2786502034 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f5ac0005400 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:13.802+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (unknown 1498667528 0 0) 0x7f5ac4015100 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:13.803+0000 7f5ad17fa700 1 -- 172.21.15.178:0/2786502034 <== mon.0 v1:172.21.15.178:6789/0 11 ==== mgrmap(e 9) v1 ==== 45910+0+0 (unknown 825629190 0 0) 0x7f5ac4045d80 con 0x7f5ad407b770 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:13.804+0000 7f5ada575700 1 -- 172.21.15.178:0/2786502034 >> v1:172.21.15.178:6800/366374299 conn(0x7f5abc03a0d0 legacy=0x7f5abc03c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:13.804+0000 7f5ada575700 1 -- 172.21.15.178:0/2786502034 >> v1:172.21.15.178:6789/0 conn(0x7f5ad407b770 legacy=0x7f5ad419b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:13.805+0000 7f5ada575700 1 -- 172.21.15.178:0/2786502034 shutdown_connections 2024-02-14T17:37:14.406 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:13.805+0000 7f5ada575700 1 -- 172.21.15.178:0/2786502034 >> 172.21.15.178:0/2786502034 conn(0x7f5ad4101040 msgr2=0x7f5ad4109f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:14.406 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:13.805+0000 7f5ada575700 1 -- 172.21.15.178:0/2786502034 shutdown_connections 2024-02-14T17:37:14.406 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:13.805+0000 7f5ada575700 1 -- 172.21.15.178:0/2786502034 wait complete. 2024-02-14T17:37:14.658 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:14 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:14.389+0000 7f918f656200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:37:14.658 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:14 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:14.489+0000 7f918f656200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:37:15.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2786502034' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-14T17:37:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:14 smithi178 ceph-mon[30321]: mgrmap e9: y(active, since 19s) 2024-02-14T17:37:15.399 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:15.154+0000 7f918f656200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:37:15.399 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:15.251+0000 7f918f656200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:37:15.399 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:15.397+0000 7f918f656200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:37:15.803 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:15.494+0000 7f918f656200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:37:15.803 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:15.578+0000 7f918f656200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:37:15.805 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:37:15.811 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-14T17:37:15.811 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T17:37:15.811 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-14T17:37:15.811 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-14T17:37:15.811 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:37:15.811 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.876+0000 7fc62d324700 1 Processor -- start 2024-02-14T17:37:15.811 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.876+0000 7fc62d324700 1 -- start start 2024-02-14T17:37:15.812 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.877+0000 7fc62d324700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc628104b90 con 0x7fc628106080 2024-02-14T17:37:15.812 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.877+0000 7fc626ffd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc628106080 0x7fc6281064a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:53334/0 (socket says 172.21.15.178:53334) 2024-02-14T17:37:15.812 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.877+0000 7fc626ffd700 1 -- 172.21.15.178:0/2569786226 learned_addr learned my addr 172.21.15.178:0/2569786226 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:15.812 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.877+0000 7fc625ffb700 1 -- 172.21.15.178:0/2569786226 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010243654 0 0) 0x7fc628104b90 con 0x7fc628106080 2024-02-14T17:37:15.812 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.877+0000 7fc625ffb700 1 -- 172.21.15.178:0/2569786226 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6100035b0 con 0x7fc628106080 2024-02-14T17:37:15.812 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.877+0000 7fc625ffb700 1 -- 172.21.15.178:0/2569786226 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2729450794 0 0) 0x7fc6100035b0 con 0x7fc628106080 2024-02-14T17:37:15.812 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.877+0000 7fc625ffb700 1 -- 172.21.15.178:0/2569786226 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc628104d20 con 0x7fc628106080 2024-02-14T17:37:15.812 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.877+0000 7fc625ffb700 1 -- 172.21.15.178:0/2569786226 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc618001580 con 0x7fc628106080 2024-02-14T17:37:15.812 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.878+0000 7fc625ffb700 1 -- 172.21.15.178:0/2569786226 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fc618001c30 con 0x7fc628106080 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.878+0000 7fc625ffb700 1 -- 172.21.15.178:0/2569786226 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc6180057e0 con 0x7fc628106080 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.878+0000 7fc62d324700 1 -- 172.21.15.178:0/2569786226 >> v1:172.21.15.178:6789/0 conn(0x7fc628106080 legacy=0x7fc6281064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.878+0000 7fc62d324700 1 -- 172.21.15.178:0/2569786226 shutdown_connections 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.878+0000 7fc62d324700 1 -- 172.21.15.178:0/2569786226 >> 172.21.15.178:0/2569786226 conn(0x7fc6280ff700 msgr2=0x7fc628101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.878+0000 7fc62d324700 1 -- 172.21.15.178:0/2569786226 shutdown_connections 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.878+0000 7fc62d324700 1 -- 172.21.15.178:0/2569786226 wait complete. 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.879+0000 7fc62d324700 1 Processor -- start 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.879+0000 7fc62d324700 1 -- start start 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.879+0000 7fc62d324700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc62819bda0 con 0x7fc628106080 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.879+0000 7fc626ffd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc628106080 0x7fc62819b690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:53336/0 (socket says 172.21.15.178:53336) 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.879+0000 7fc626ffd700 1 -- 172.21.15.178:0/4251698124 learned_addr learned my addr 172.21.15.178:0/4251698124 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.879+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920829153 0 0) 0x7fc62819bda0 con 0x7fc628106080 2024-02-14T17:37:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.879+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc610005540 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.880+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1165311876 0 0) 0x7fc610005540 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.880+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc62819bda0 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.880+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc618000ed0 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.880+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 427485524 0 0) 0x7fc62819bda0 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.880+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62819bf30 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.880+0000 7fc62d324700 1 -- 172.21.15.178:0/4251698124 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62819c1e0 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.880+0000 7fc62d324700 1 -- 172.21.15.178:0/4251698124 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62819fd30 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.880+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fc6180049d0 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.880+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc618006160 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.881+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 9) v1 ==== 45910+0+0 (unknown 825629190 0 0) 0x7fc618011640 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.881+0000 7fc62d324700 1 -- 172.21.15.178:0/4251698124 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc614005150 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.881+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7fc618047390 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.881+0000 7fc6267fc700 1 -- 172.21.15.178:0/4251698124 >> v1:172.21.15.178:6800/366374299 conn(0x7fc61003a1b0 legacy=0x7fc61003c670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/366374299 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:14.886+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fc618012f40 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:15.082+0000 7fc6267fc700 1 -- 172.21.15.178:0/4251698124 >> v1:172.21.15.178:6800/366374299 conn(0x7fc61003a1b0 legacy=0x7fc61003c670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/366374299 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:15.140+0000 7fc62d324700 1 -- 172.21.15.178:0/4251698124 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc614005c10 con 0x7fc628106080 2024-02-14T17:37:15.814 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:15.141+0000 7fc60ffff700 1 -- 172.21.15.178:0/4251698124 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (unknown 2005831594 0 1748205903) 0x7fc618015020 con 0x7fc628106080 2024-02-14T17:37:15.815 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:15.143+0000 7fc62d324700 1 -- 172.21.15.178:0/4251698124 >> v1:172.21.15.178:6800/366374299 conn(0x7fc61003a1b0 legacy=0x7fc61003c670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:37:15.815 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:15.143+0000 7fc62d324700 1 -- 172.21.15.178:0/4251698124 >> v1:172.21.15.178:6789/0 conn(0x7fc628106080 legacy=0x7fc62819b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:15.815 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:15.143+0000 7fc62d324700 1 -- 172.21.15.178:0/4251698124 shutdown_connections 2024-02-14T17:37:15.815 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:15.143+0000 7fc62d324700 1 -- 172.21.15.178:0/4251698124 >> 172.21.15.178:0/4251698124 conn(0x7fc6280ff700 msgr2=0x7fc628101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:15.815 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:15.143+0000 7fc62d324700 1 -- 172.21.15.178:0/4251698124 shutdown_connections 2024-02-14T17:37:15.815 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:15.143+0000 7fc62d324700 1 -- 172.21.15.178:0/4251698124 wait complete. 2024-02-14T17:37:15.815 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for the mgr to restart... 2024-02-14T17:37:15.815 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mgr epoch 9... 2024-02-14T17:37:16.078 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4251698124' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-14T17:37:16.078 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:15.853+0000 7f918f656200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:37:16.078 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:15.965+0000 7f918f656200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:37:16.078 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:16 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:16.051+0000 7f918f656200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:37:16.329 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:16 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:16.153+0000 7f918f656200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:37:16.583 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:16 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:16.338+0000 7f918f656200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:37:17.582 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:17 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:17.305+0000 7f918f656200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:37:17.583 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:17 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:17.436+0000 7f918f656200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:37:18.056 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:17 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:17.771+0000 7f918f656200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:37:18.056 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:17 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:17.861+0000 7f918f656200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:37:18.057 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:18 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:18.054+0000 7f918f656200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:37:18.333 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:18 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:18.241+0000 7f918f656200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:37:18.791 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:18 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:18.527+0000 7f918f656200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:37:18.791 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:18 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:18.789+0000 7f918f656200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:37:19.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:19 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:19.026+0000 7f918f656200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:37:20.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:19 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:19.827+0000 7f918f656200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:37:20.717 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:20.397+0000 7f918f656200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:37:20.718 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:20.496+0000 7f918f656200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:37:20.718 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:37:20.716+0000 7f918f656200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:37:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: Active manager daemon y restarted 2024-02-14T17:37:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: Activating manager daemon y 2024-02-14T17:37:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: osdmap e3: 0 total, 0 up, 0 in 2024-02-14T17:37:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: mgrmap e10: y(active, starting, since 0.00323639s) 2024-02-14T17:37:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:37:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:37:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:37:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:37:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:37:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: Manager daemon y is now available 2024-02-14T17:37:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:37:21.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:20 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:37:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:21 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:37:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:21 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:37:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:21 smithi178 ceph-mon[30321]: mgrmap e11: y(active, since 1.00619s) 2024-02-14T17:37:22.394 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:37:22.394 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.327+0000 7f8d6d570700 1 Processor -- start 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.327+0000 7f8d6d570700 1 -- start start 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.327+0000 7f8d6d570700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8d68079720 con 0x7f8d681065d0 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.328+0000 7f8d66ffd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f8d681065d0 0x7f8d681089c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:53340/0 (socket says 172.21.15.178:53340) 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.328+0000 7f8d66ffd700 1 -- 172.21.15.178:0/2711178936 learned_addr learned my addr 172.21.15.178:0/2711178936 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.328+0000 7f8d65ffb700 1 -- 172.21.15.178:0/2711178936 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350848612 0 0) 0x7f8d68079720 con 0x7f8d681065d0 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.328+0000 7f8d65ffb700 1 -- 172.21.15.178:0/2711178936 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d500035b0 con 0x7f8d681065d0 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.328+0000 7f8d65ffb700 1 -- 172.21.15.178:0/2711178936 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 159403359 0 0) 0x7f8d500035b0 con 0x7f8d681065d0 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.328+0000 7f8d65ffb700 1 -- 172.21.15.178:0/2711178936 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6810b0f0 con 0x7f8d681065d0 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.328+0000 7f8d65ffb700 1 -- 172.21.15.178:0/2711178936 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f8d5c001580 con 0x7f8d681065d0 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.328+0000 7f8d65ffb700 1 -- 172.21.15.178:0/2711178936 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f8d5c001d00 con 0x7f8d681065d0 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.328+0000 7f8d65ffb700 1 -- 172.21.15.178:0/2711178936 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f8d5c005960 con 0x7f8d681065d0 2024-02-14T17:37:22.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.329+0000 7f8d6d570700 1 -- 172.21.15.178:0/2711178936 >> v1:172.21.15.178:6789/0 conn(0x7f8d681065d0 legacy=0x7f8d681089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.329+0000 7f8d6d570700 1 -- 172.21.15.178:0/2711178936 shutdown_connections 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.329+0000 7f8d6d570700 1 -- 172.21.15.178:0/2711178936 >> 172.21.15.178:0/2711178936 conn(0x7f8d680fffa0 msgr2=0x7f8d68102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.329+0000 7f8d6d570700 1 -- 172.21.15.178:0/2711178936 shutdown_connections 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.329+0000 7f8d6d570700 1 -- 172.21.15.178:0/2711178936 wait complete. 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.330+0000 7f8d6d570700 1 Processor -- start 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.330+0000 7f8d6d570700 1 -- start start 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.330+0000 7f8d6d570700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8d68199b70 con 0x7f8d681065d0 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.330+0000 7f8d66ffd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f8d681065d0 0x7f8d68199460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:53354/0 (socket says 172.21.15.178:53354) 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.330+0000 7f8d66ffd700 1 -- 172.21.15.178:0/3437362065 learned_addr learned my addr 172.21.15.178:0/3437362065 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.330+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244887881 0 0) 0x7f8d68199b70 con 0x7f8d681065d0 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.330+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d500054f0 con 0x7f8d681065d0 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.331+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3319979965 0 0) 0x7f8d500054f0 con 0x7f8d681065d0 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.331+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d68199b70 con 0x7f8d681065d0 2024-02-14T17:37:22.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.331+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f8d5c001080 con 0x7f8d681065d0 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.331+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2156158297 0 0) 0x7f8d68199b70 con 0x7f8d681065d0 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.331+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68199d00 con 0x7f8d681065d0 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.331+0000 7f8d6d570700 1 -- 172.21.15.178:0/3437362065 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d68199fb0 con 0x7f8d681065d0 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.331+0000 7f8d6d570700 1 -- 172.21.15.178:0/3437362065 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6819db00 con 0x7f8d681065d0 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.331+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f8d5c0059b0 con 0x7f8d681065d0 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.331+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f8d5c006260 con 0x7f8d681065d0 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.332+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 9) v1 ==== 45910+0+0 (unknown 825629190 0 0) 0x7f8d5c0117e0 con 0x7f8d681065d0 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.332+0000 7f8d6d570700 1 -- 172.21.15.178:0/3437362065 --> v1:172.21.15.178:6800/366374299 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d54000d40 con 0x7f8d5003a130 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.332+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2702214437 0 0) 0x7f8d5c048360 con 0x7f8d681065d0 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.332+0000 7f8d667fc700 1 -- 172.21.15.178:0/3437362065 >> v1:172.21.15.178:6800/366374299 conn(0x7f8d5003a130 legacy=0x7f8d5003c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/366374299 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.533+0000 7f8d667fc700 1 -- 172.21.15.178:0/3437362065 >> v1:172.21.15.178:6800/366374299 conn(0x7f8d5003a130 legacy=0x7f8d5003c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/366374299 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:16.934+0000 7f8d667fc700 1 -- 172.21.15.178:0/3437362065 >> v1:172.21.15.178:6800/366374299 conn(0x7f8d5003a130 legacy=0x7f8d5003c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/366374299 2024-02-14T17:37:22.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:17.735+0000 7f8d667fc700 1 -- 172.21.15.178:0/3437362065 >> v1:172.21.15.178:6800/366374299 conn(0x7f8d5003a130 legacy=0x7f8d5003c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/366374299 2024-02-14T17:37:22.398 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:19.336+0000 7f8d667fc700 1 -- 172.21.15.178:0/3437362065 >> v1:172.21.15.178:6800/366374299 conn(0x7f8d5003a130 legacy=0x7f8d5003c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/366374299 2024-02-14T17:37:22.398 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:20.731+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mgrmap(e 10) v1 ==== 45712+0+0 (unknown 3910061321 0 0) 0x7f8d5c013620 con 0x7f8d681065d0 2024-02-14T17:37:22.398 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:20.731+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 >> v1:172.21.15.178:6800/366374299 conn(0x7f8d5003a130 legacy=0x7f8d5003c5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:37:22.398 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:21.733+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mgrmap(e 11) v1 ==== 45804+0+0 (unknown 3406971233 0 0) 0x7f8d5c048f90 con 0x7f8d681065d0 2024-02-14T17:37:22.398 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:21.733+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 --> v1:172.21.15.178:6800/1500761607 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d54000d40 con 0x7f8d5003ead0 2024-02-14T17:37:22.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:21.734+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (unknown 0 0 4109177037) 0x7f8d54000d40 con 0x7f8d5003ead0 2024-02-14T17:37:22.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:21.741+0000 7f8d6d570700 1 -- 172.21.15.178:0/3437362065 --> v1:172.21.15.178:6800/1500761607 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f8d54002800 con 0x7f8d5003ead0 2024-02-14T17:37:22.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:21.741+0000 7f8d4ffff700 1 -- 172.21.15.178:0/3437362065 <== mgr.14150 v1:172.21.15.178:6800/1500761607 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (unknown 0 0 3086460295) 0x7f8d54002800 con 0x7f8d5003ead0 2024-02-14T17:37:22.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:21.741+0000 7f8d6d570700 1 -- 172.21.15.178:0/3437362065 >> v1:172.21.15.178:6800/1500761607 conn(0x7f8d5003ead0 legacy=0x7f8d50040ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:22.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:21.741+0000 7f8d6d570700 1 -- 172.21.15.178:0/3437362065 >> v1:172.21.15.178:6789/0 conn(0x7f8d681065d0 legacy=0x7f8d68199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:22.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:21.741+0000 7f8d6d570700 1 -- 172.21.15.178:0/3437362065 shutdown_connections 2024-02-14T17:37:22.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:21.741+0000 7f8d6d570700 1 -- 172.21.15.178:0/3437362065 >> 172.21.15.178:0/3437362065 conn(0x7f8d680fffa0 msgr2=0x7f8d681902b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:22.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:21.741+0000 7f8d6d570700 1 -- 172.21.15.178:0/3437362065 shutdown_connections 2024-02-14T17:37:22.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:21.741+0000 7f8d6d570700 1 -- 172.21.15.178:0/3437362065 wait complete. 2024-02-14T17:37:22.399 INFO:teuthology.orchestra.run.smithi178.stdout:mgr epoch 9 is available 2024-02-14T17:37:22.399 INFO:teuthology.orchestra.run.smithi178.stdout:Generating a dashboard self-signed certificate... 2024-02-14T17:37:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:22 smithi178 ceph-mon[30321]: [14/Feb/2024:17:37:21] ENGINE Bus STARTING 2024-02-14T17:37:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:22 smithi178 ceph-mon[30321]: [14/Feb/2024:17:37:21] ENGINE Serving on http://172.21.15.178:8765 2024-02-14T17:37:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:22 smithi178 ceph-mon[30321]: [14/Feb/2024:17:37:21] ENGINE Serving on https://172.21.15.178:7150 2024-02-14T17:37:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:22 smithi178 ceph-mon[30321]: [14/Feb/2024:17:37:21] ENGINE Bus STARTED 2024-02-14T17:37:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:22 smithi178 ceph-mon[30321]: from='client.14154 v1:172.21.15.178:0/3437362065' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-14T17:37:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:22 smithi178 ceph-mon[30321]: from='client.14154 v1:172.21.15.178:0/3437362065' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-14T17:37:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:24.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:23 smithi178 ceph-mon[30321]: mgrmap e12: y(active, since 2s) 2024-02-14T17:37:24.412 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-14T17:37:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.235+0000 7f613ab0a700 1 Processor -- start 2024-02-14T17:37:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.235+0000 7f613ab0a700 1 -- start start 2024-02-14T17:37:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.235+0000 7f613ab0a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6134109f40 con 0x7f6134105b10 2024-02-14T17:37:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.235+0000 7f61388a6700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f6134105b10 0x7f6134105f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:53420/0 (socket says 172.21.15.178:53420) 2024-02-14T17:37:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.235+0000 7f61388a6700 1 -- 172.21.15.178:0/3766881400 learned_addr learned my addr 172.21.15.178:0/3766881400 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.235+0000 7f61337fe700 1 -- 172.21.15.178:0/3766881400 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091234098 0 0) 0x7f6134109f40 con 0x7f6134105b10 2024-02-14T17:37:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.235+0000 7f61337fe700 1 -- 172.21.15.178:0/3766881400 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61240035b0 con 0x7f6134105b10 2024-02-14T17:37:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.236+0000 7f61337fe700 1 -- 172.21.15.178:0/3766881400 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2555003101 0 0) 0x7f61240035b0 con 0x7f6134105b10 2024-02-14T17:37:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.236+0000 7f61337fe700 1 -- 172.21.15.178:0/3766881400 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613410b0e0 con 0x7f6134105b10 2024-02-14T17:37:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.236+0000 7f61337fe700 1 -- 172.21.15.178:0/3766881400 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f6128001580 con 0x7f6134105b10 2024-02-14T17:37:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.236+0000 7f61337fe700 1 -- 172.21.15.178:0/3766881400 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f6128001d00 con 0x7f6134105b10 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.236+0000 7f61337fe700 1 -- 172.21.15.178:0/3766881400 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f6128005960 con 0x7f6134105b10 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.236+0000 7f613ab0a700 1 -- 172.21.15.178:0/3766881400 >> v1:172.21.15.178:6789/0 conn(0x7f6134105b10 legacy=0x7f6134105f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.236+0000 7f613ab0a700 1 -- 172.21.15.178:0/3766881400 shutdown_connections 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.236+0000 7f613ab0a700 1 -- 172.21.15.178:0/3766881400 >> 172.21.15.178:0/3766881400 conn(0x7f61341010b0 msgr2=0x7f61341034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.236+0000 7f613ab0a700 1 -- 172.21.15.178:0/3766881400 shutdown_connections 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.236+0000 7f613ab0a700 1 -- 172.21.15.178:0/3766881400 wait complete. 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.237+0000 7f613ab0a700 1 Processor -- start 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.237+0000 7f613ab0a700 1 -- start start 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.237+0000 7f613ab0a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6134199b00 con 0x7f6134105b10 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.237+0000 7f61388a6700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f6134105b10 0x7f61341993f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:53426/0 (socket says 172.21.15.178:53426) 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.237+0000 7f61388a6700 1 -- 172.21.15.178:0/3496620032 learned_addr learned my addr 172.21.15.178:0/3496620032 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.237+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147980296 0 0) 0x7f6134199b00 con 0x7f6134105b10 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.237+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61240054f0 con 0x7f6134105b10 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.237+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1361341698 0 0) 0x7f61240054f0 con 0x7f6134105b10 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.237+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6134199b00 con 0x7f6134105b10 2024-02-14T17:37:24.414 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.237+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f6128001080 con 0x7f6134105b10 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.238+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 17221215 0 0) 0x7f6134199b00 con 0x7f6134105b10 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.238+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6134199c90 con 0x7f6134105b10 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.238+0000 7f613ab0a700 1 -- 172.21.15.178:0/3496620032 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6134199f40 con 0x7f6134105b10 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.238+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f61280049f0 con 0x7f6134105b10 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.238+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f61280062a0 con 0x7f6134105b10 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.238+0000 7f613ab0a700 1 -- 172.21.15.178:0/3496620032 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f613419da90 con 0x7f6134105b10 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.238+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 12) v1 ==== 45910+0+0 (unknown 2056327466 0 0) 0x7f61280118d0 con 0x7f6134105b10 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.239+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843057982 0 0) 0x7f61280478e0 con 0x7f6134105b10 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.239+0000 7f613ab0a700 1 -- 172.21.15.178:0/3496620032 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6134192a70 con 0x7f6134105b10 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.244+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f6128016ce0 con 0x7f6134105b10 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.448+0000 7f613ab0a700 1 -- 172.21.15.178:0/3496620032 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f61340620b0 con 0x7f612403a170 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.815+0000 7f6131ffb700 1 -- 172.21.15.178:0/3496620032 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (unknown 0 0 3317752739) 0x7f61340620b0 con 0x7f612403a170 2024-02-14T17:37:24.415 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.817+0000 7f613ab0a700 1 -- 172.21.15.178:0/3496620032 >> v1:172.21.15.178:6800/1500761607 conn(0x7f612403a170 legacy=0x7f612403c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:24.416 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.817+0000 7f613ab0a700 1 -- 172.21.15.178:0/3496620032 >> v1:172.21.15.178:6789/0 conn(0x7f6134105b10 legacy=0x7f61341993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:24.416 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.817+0000 7f613ab0a700 1 -- 172.21.15.178:0/3496620032 shutdown_connections 2024-02-14T17:37:24.416 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.817+0000 7f613ab0a700 1 -- 172.21.15.178:0/3496620032 >> 172.21.15.178:0/3496620032 conn(0x7f61341010b0 msgr2=0x7f61341034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:24.416 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.818+0000 7f613ab0a700 1 -- 172.21.15.178:0/3496620032 shutdown_connections 2024-02-14T17:37:24.416 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:23.818+0000 7f613ab0a700 1 -- 172.21.15.178:0/3496620032 wait complete. 2024-02-14T17:37:24.416 INFO:teuthology.orchestra.run.smithi178.stdout:Creating initial admin user... 2024-02-14T17:37:25.012 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:24 smithi178 ceph-mon[30321]: from='client.14162 v1:172.21.15.178:0/3496620032' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:37:25.012 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:24 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:25.012 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:24 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:25.013 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:24 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$RR5SHrrq2AkwCkcC3xIJAuAsiFT6.6jOpjaqcjyzlN6DTx7HvPcUO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707932245, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.847+0000 7fa064d16700 1 Processor -- start 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.847+0000 7fa064d16700 1 -- start start 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.847+0000 7fa064d16700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa060105d60 con 0x7fa060103e00 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.847+0000 7fa05e59c700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa060103e00 0x7fa060104220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:55154/0 (socket says 172.21.15.178:55154) 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.847+0000 7fa05e59c700 1 -- 172.21.15.178:0/3210778872 learned_addr learned my addr 172.21.15.178:0/3210778872 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.848+0000 7fa05d59a700 1 -- 172.21.15.178:0/3210778872 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429299841 0 0) 0x7fa060105d60 con 0x7fa060103e00 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.848+0000 7fa05d59a700 1 -- 172.21.15.178:0/3210778872 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0480035b0 con 0x7fa060103e00 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.848+0000 7fa05d59a700 1 -- 172.21.15.178:0/3210778872 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449436560 0 0) 0x7fa0480035b0 con 0x7fa060103e00 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.848+0000 7fa05d59a700 1 -- 172.21.15.178:0/3210778872 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa060105ef0 con 0x7fa060103e00 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.848+0000 7fa05d59a700 1 -- 172.21.15.178:0/3210778872 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fa050001580 con 0x7fa060103e00 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.848+0000 7fa05d59a700 1 -- 172.21.15.178:0/3210778872 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fa050001d00 con 0x7fa060103e00 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.848+0000 7fa05d59a700 1 -- 172.21.15.178:0/3210778872 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fa050005960 con 0x7fa060103e00 2024-02-14T17:37:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.849+0000 7fa064d16700 1 -- 172.21.15.178:0/3210778872 >> v1:172.21.15.178:6789/0 conn(0x7fa060103e00 legacy=0x7fa060104220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.849+0000 7fa064d16700 1 -- 172.21.15.178:0/3210778872 shutdown_connections 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.849+0000 7fa064d16700 1 -- 172.21.15.178:0/3210778872 >> 172.21.15.178:0/3210778872 conn(0x7fa0600ff730 msgr2=0x7fa060101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.849+0000 7fa064d16700 1 -- 172.21.15.178:0/3210778872 shutdown_connections 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.849+0000 7fa064d16700 1 -- 172.21.15.178:0/3210778872 wait complete. 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.850+0000 7fa064d16700 1 Processor -- start 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.850+0000 7fa064d16700 1 -- start start 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.850+0000 7fa064d16700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa06019bd50 con 0x7fa060103e00 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.850+0000 7fa05e59c700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa060103e00 0x7fa06019b640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:55158/0 (socket says 172.21.15.178:55158) 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.850+0000 7fa05e59c700 1 -- 172.21.15.178:0/1734628596 learned_addr learned my addr 172.21.15.178:0/1734628596 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.850+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837184957 0 0) 0x7fa06019bd50 con 0x7fa060103e00 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.850+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0480054f0 con 0x7fa060103e00 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.850+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3158683331 0 0) 0x7fa0480054f0 con 0x7fa060103e00 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.850+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa06019bd50 con 0x7fa060103e00 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.850+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fa050001080 con 0x7fa060103e00 2024-02-14T17:37:26.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.851+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3792989144 0 0) 0x7fa06019bd50 con 0x7fa060103e00 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.851+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06019bee0 con 0x7fa060103e00 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.851+0000 7fa064d16700 1 -- 172.21.15.178:0/1734628596 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06019c190 con 0x7fa060103e00 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.851+0000 7fa064d16700 1 -- 172.21.15.178:0/1734628596 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06019fce0 con 0x7fa060103e00 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.851+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fa050004a60 con 0x7fa060103e00 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.851+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fa050006160 con 0x7fa060103e00 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.852+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 12) v1 ==== 45910+0+0 (unknown 2056327466 0 0) 0x7fa050011640 con 0x7fa060103e00 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.852+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843057982 0 0) 0x7fa050048210 con 0x7fa060103e00 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.852+0000 7fa064d16700 1 -- 172.21.15.178:0/1734628596 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa040005150 con 0x7fa060103e00 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:24.857+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fa050012eb0 con 0x7fa060103e00 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:25.090+0000 7fa064d16700 1 -- 172.21.15.178:0/1734628596 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa040002430 con 0x7fa04803a210 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:25.314+0000 7fa04f7fe700 1 -- 172.21.15.178:0/1734628596 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (unknown 0 0 1203546306) 0x7fa040002430 con 0x7fa04803a210 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:25.316+0000 7fa064d16700 1 -- 172.21.15.178:0/1734628596 >> v1:172.21.15.178:6800/1500761607 conn(0x7fa04803a210 legacy=0x7fa04803c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:25.316+0000 7fa064d16700 1 -- 172.21.15.178:0/1734628596 >> v1:172.21.15.178:6789/0 conn(0x7fa060103e00 legacy=0x7fa06019b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:26.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:25.316+0000 7fa064d16700 1 -- 172.21.15.178:0/1734628596 shutdown_connections 2024-02-14T17:37:26.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:25.317+0000 7fa064d16700 1 -- 172.21.15.178:0/1734628596 >> 172.21.15.178:0/1734628596 conn(0x7fa0600ff730 msgr2=0x7fa060101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:26.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:25.317+0000 7fa064d16700 1 -- 172.21.15.178:0/1734628596 shutdown_connections 2024-02-14T17:37:26.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:25.317+0000 7fa064d16700 1 -- 172.21.15.178:0/1734628596 wait complete. 2024-02-14T17:37:26.317 INFO:teuthology.orchestra.run.smithi178.stdout:Fetching dashboard port number... 2024-02-14T17:37:26.575 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:26 smithi178 ceph-mon[30321]: from='client.14164 v1:172.21.15.178:0/1734628596' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:37:26.576 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:26 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:27.922 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 8443 2024-02-14T17:37:27.922 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.139+0000 7fc70a1f6700 1 Processor -- start 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.139+0000 7fc70a1f6700 1 -- start start 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.140+0000 7fc70a1f6700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc70407a370 con 0x7fc70407b770 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.140+0000 7fc7037fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc70407b770 0x7fc704079bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:55172/0 (socket says 172.21.15.178:55172) 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.140+0000 7fc7037fe700 1 -- 172.21.15.178:0/2972427758 learned_addr learned my addr 172.21.15.178:0/2972427758 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.140+0000 7fc702ffd700 1 -- 172.21.15.178:0/2972427758 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403508742 0 0) 0x7fc70407a370 con 0x7fc70407b770 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.140+0000 7fc702ffd700 1 -- 172.21.15.178:0/2972427758 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f00035b0 con 0x7fc70407b770 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.140+0000 7fc702ffd700 1 -- 172.21.15.178:0/2972427758 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1352290241 0 0) 0x7fc6f00035b0 con 0x7fc70407b770 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.140+0000 7fc702ffd700 1 -- 172.21.15.178:0/2972427758 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70407a500 con 0x7fc70407b770 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.140+0000 7fc702ffd700 1 -- 172.21.15.178:0/2972427758 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc6f4001580 con 0x7fc70407b770 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.140+0000 7fc702ffd700 1 -- 172.21.15.178:0/2972427758 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fc6f4001d00 con 0x7fc70407b770 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.140+0000 7fc702ffd700 1 -- 172.21.15.178:0/2972427758 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc6f4005960 con 0x7fc70407b770 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.141+0000 7fc70a1f6700 1 -- 172.21.15.178:0/2972427758 >> v1:172.21.15.178:6789/0 conn(0x7fc70407b770 legacy=0x7fc704079bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.141+0000 7fc70a1f6700 1 -- 172.21.15.178:0/2972427758 shutdown_connections 2024-02-14T17:37:27.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.141+0000 7fc70a1f6700 1 -- 172.21.15.178:0/2972427758 >> 172.21.15.178:0/2972427758 conn(0x7fc7041010d0 msgr2=0x7fc7041034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.141+0000 7fc70a1f6700 1 -- 172.21.15.178:0/2972427758 shutdown_connections 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.141+0000 7fc70a1f6700 1 -- 172.21.15.178:0/2972427758 wait complete. 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.141+0000 7fc70a1f6700 1 Processor -- start 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.141+0000 7fc70a1f6700 1 -- start start 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.142+0000 7fc70a1f6700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc70419bda0 con 0x7fc70407b770 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.142+0000 7fc7037fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc70407b770 0x7fc70419b690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:55180/0 (socket says 172.21.15.178:55180) 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.142+0000 7fc7037fe700 1 -- 172.21.15.178:0/4233492153 learned_addr learned my addr 172.21.15.178:0/4233492153 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.142+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202041682 0 0) 0x7fc70419bda0 con 0x7fc70407b770 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.142+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f00054f0 con 0x7fc70407b770 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.142+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3646807718 0 0) 0x7fc6f00054f0 con 0x7fc70407b770 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.142+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc70419bda0 con 0x7fc70407b770 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.142+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc6f4001080 con 0x7fc70407b770 2024-02-14T17:37:27.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.142+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2941826614 0 0) 0x7fc70419bda0 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.142+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70419bf30 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.142+0000 7fc70a1f6700 1 -- 172.21.15.178:0/4233492153 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70419c1e0 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.143+0000 7fc70a1f6700 1 -- 172.21.15.178:0/4233492153 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70419fd30 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.143+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fc6f4004a60 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.143+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fc6f4006210 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.143+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 12) v1 ==== 45910+0+0 (unknown 2056327466 0 0) 0x7fc6f40117b0 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.143+0000 7fc70a1f6700 1 -- 172.21.15.178:0/4233492153 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e4005150 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.144+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843057982 0 0) 0x7fc6f4047570 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.148+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fc6f4013070 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.315+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7fc6f4045e10 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.342+0000 7fc70a1f6700 1 -- 172.21.15.178:0/4233492153 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fc6e4005980 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.342+0000 7fc7017fa700 1 -- 172.21.15.178:0/4233492153 <== mon.0 v1:172.21.15.178:6789/0 11 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (unknown 3713421687 0 83753974) 0x7fc6f4013070 con 0x7fc70407b770 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.345+0000 7fc70a1f6700 1 -- 172.21.15.178:0/4233492153 >> v1:172.21.15.178:6800/1500761607 conn(0x7fc6f003a120 legacy=0x7fc6f003c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.345+0000 7fc70a1f6700 1 -- 172.21.15.178:0/4233492153 >> v1:172.21.15.178:6789/0 conn(0x7fc70407b770 legacy=0x7fc70419b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.345+0000 7fc70a1f6700 1 -- 172.21.15.178:0/4233492153 shutdown_connections 2024-02-14T17:37:27.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.345+0000 7fc70a1f6700 1 -- 172.21.15.178:0/4233492153 >> 172.21.15.178:0/4233492153 conn(0x7fc7041010d0 msgr2=0x7fc70410a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:27.926 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.345+0000 7fc70a1f6700 1 -- 172.21.15.178:0/4233492153 shutdown_connections 2024-02-14T17:37:27.926 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:27.345+0000 7fc70a1f6700 1 -- 172.21.15.178:0/4233492153 wait complete. 2024-02-14T17:37:27.929 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:37:27.929 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout disabled 2024-02-14T17:37:27.935 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:37:27.935 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2024-02-14T17:37:27.935 INFO:teuthology.orchestra.run.smithi178.stdout:firewalld.service is not enabled 2024-02-14T17:37:27.935 INFO:teuthology.orchestra.run.smithi178.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-14T17:37:27.938 INFO:teuthology.orchestra.run.smithi178.stdout:Ceph Dashboard is now available at: 2024-02-14T17:37:27.938 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:37:27.938 INFO:teuthology.orchestra.run.smithi178.stdout: URL: https://smithi178.front.sepia.ceph.com:8443/ 2024-02-14T17:37:27.938 INFO:teuthology.orchestra.run.smithi178.stdout: User: admin 2024-02-14T17:37:27.938 INFO:teuthology.orchestra.run.smithi178.stdout: Password: p4fzaiujbw 2024-02-14T17:37:27.938 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:37:27.938 INFO:teuthology.orchestra.run.smithi178.stdout:Saving cluster configuration to /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config directory 2024-02-14T17:37:27.938 INFO:teuthology.orchestra.run.smithi178.stdout:Enabling autotune for osd_memory_target 2024-02-14T17:37:28.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:28 smithi178 ceph-mon[30321]: mgrmap e13: y(active, since 6s) 2024-02-14T17:37:28.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4233492153' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-14T17:37:29.287 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.455+0000 7f37112ce700 1 Processor -- start 2024-02-14T17:37:29.287 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.455+0000 7f37112ce700 1 -- start start 2024-02-14T17:37:29.287 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.455+0000 7f37112ce700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f370c078e90 con 0x7f370c1043a0 2024-02-14T17:37:29.287 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.455+0000 7f370affd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f370c1043a0 0x7f370c1047c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:55196/0 (socket says 172.21.15.178:55196) 2024-02-14T17:37:29.287 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.455+0000 7f370affd700 1 -- 172.21.15.178:0/2973439007 learned_addr learned my addr 172.21.15.178:0/2973439007 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:29.287 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.455+0000 7f3709ffb700 1 -- 172.21.15.178:0/2973439007 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390542194 0 0) 0x7f370c078e90 con 0x7f370c1043a0 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.455+0000 7f3709ffb700 1 -- 172.21.15.178:0/2973439007 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36f40035b0 con 0x7f370c1043a0 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.455+0000 7f3709ffb700 1 -- 172.21.15.178:0/2973439007 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1836288673 0 0) 0x7f36f40035b0 con 0x7f370c1043a0 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.455+0000 7f3709ffb700 1 -- 172.21.15.178:0/2973439007 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c105ee0 con 0x7f370c1043a0 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.456+0000 7f3709ffb700 1 -- 172.21.15.178:0/2973439007 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f36fc001580 con 0x7f370c1043a0 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.456+0000 7f3709ffb700 1 -- 172.21.15.178:0/2973439007 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f36fc001d00 con 0x7f370c1043a0 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.456+0000 7f3709ffb700 1 -- 172.21.15.178:0/2973439007 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f36fc005960 con 0x7f370c1043a0 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.456+0000 7f37112ce700 1 -- 172.21.15.178:0/2973439007 >> v1:172.21.15.178:6789/0 conn(0x7f370c1043a0 legacy=0x7f370c1047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.456+0000 7f37112ce700 1 -- 172.21.15.178:0/2973439007 shutdown_connections 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.456+0000 7f37112ce700 1 -- 172.21.15.178:0/2973439007 >> 172.21.15.178:0/2973439007 conn(0x7f370c0fff80 msgr2=0x7f370c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.456+0000 7f37112ce700 1 -- 172.21.15.178:0/2973439007 shutdown_connections 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.456+0000 7f37112ce700 1 -- 172.21.15.178:0/2973439007 wait complete. 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.457+0000 7f37112ce700 1 Processor -- start 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.457+0000 7f37112ce700 1 -- start start 2024-02-14T17:37:29.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.457+0000 7f37112ce700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f370c19bd50 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.457+0000 7f370affd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f370c1043a0 0x7f370c19b640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:55198/0 (socket says 172.21.15.178:55198) 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.457+0000 7f370affd700 1 -- 172.21.15.178:0/1336330563 learned_addr learned my addr 172.21.15.178:0/1336330563 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.457+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326955433 0 0) 0x7f370c19bd50 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.457+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36f40054f0 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.457+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 576613059 0 0) 0x7f36f40054f0 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.458+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f370c19bd50 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.458+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f36fc001080 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.458+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2281838501 0 0) 0x7f370c19bd50 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.458+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c19bee0 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.458+0000 7f37112ce700 1 -- 172.21.15.178:0/1336330563 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c19c190 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.458+0000 7f37112ce700 1 -- 172.21.15.178:0/1336330563 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c19fce0 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.458+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f36fc004a60 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.458+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f36fc006190 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.458+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7f36fc0116a0 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.459+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843057982 0 0) 0x7f36fc048320 con 0x7f370c1043a0 2024-02-14T17:37:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.459+0000 7f37112ce700 1 -- 172.21.15.178:0/1336330563 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f8005150 con 0x7f370c1043a0 2024-02-14T17:37:29.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.464+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f36fc012fc0 con 0x7f370c1043a0 2024-02-14T17:37:29.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.659+0000 7f37112ce700 1 -- 172.21.15.178:0/1336330563 --> v1:172.21.15.178:6789/0 -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f36f8005980 con 0x7f370c1043a0 2024-02-14T17:37:29.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.659+0000 7f36f3fff700 1 -- 172.21.15.178:0/1336330563 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (unknown 689008156 0 0) 0x7f36fc047c10 con 0x7f370c1043a0 2024-02-14T17:37:29.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.661+0000 7f37112ce700 1 -- 172.21.15.178:0/1336330563 >> v1:172.21.15.178:6800/1500761607 conn(0x7f36f403a260 legacy=0x7f36f403c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:29.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.661+0000 7f37112ce700 1 -- 172.21.15.178:0/1336330563 >> v1:172.21.15.178:6789/0 conn(0x7f370c1043a0 legacy=0x7f370c19b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:29.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.661+0000 7f37112ce700 1 -- 172.21.15.178:0/1336330563 shutdown_connections 2024-02-14T17:37:29.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.661+0000 7f37112ce700 1 -- 172.21.15.178:0/1336330563 >> 172.21.15.178:0/1336330563 conn(0x7f370c0fff80 msgr2=0x7f370c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:29.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.661+0000 7f37112ce700 1 -- 172.21.15.178:0/1336330563 shutdown_connections 2024-02-14T17:37:29.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:28.661+0000 7f37112ce700 1 -- 172.21.15.178:0/1336330563 wait complete. 2024-02-14T17:37:30.601 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.742+0000 7ff4214e5700 1 Processor -- start 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.742+0000 7ff4214e5700 1 -- start start 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.742+0000 7ff4214e5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff41c108e40 con 0x7ff41c104a10 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.742+0000 7ff41affd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff41c104a10 0x7ff41c104e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:55212/0 (socket says 172.21.15.178:55212) 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.742+0000 7ff41affd700 1 -- 172.21.15.178:0/4191251282 learned_addr learned my addr 172.21.15.178:0/4191251282 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.742+0000 7ff419ffb700 1 -- 172.21.15.178:0/4191251282 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178960836 0 0) 0x7ff41c108e40 con 0x7ff41c104a10 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.742+0000 7ff419ffb700 1 -- 172.21.15.178:0/4191251282 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4040035b0 con 0x7ff41c104a10 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.743+0000 7ff419ffb700 1 -- 172.21.15.178:0/4191251282 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1500183382 0 0) 0x7ff4040035b0 con 0x7ff41c104a10 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.743+0000 7ff419ffb700 1 -- 172.21.15.178:0/4191251282 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c109fe0 con 0x7ff41c104a10 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.743+0000 7ff419ffb700 1 -- 172.21.15.178:0/4191251282 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7ff40c001580 con 0x7ff41c104a10 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.743+0000 7ff419ffb700 1 -- 172.21.15.178:0/4191251282 <== mon.0 v1:172.21.15.178:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7ff40c001d00 con 0x7ff41c104a10 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.743+0000 7ff419ffb700 1 -- 172.21.15.178:0/4191251282 <== mon.0 v1:172.21.15.178:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7ff40c005960 con 0x7ff41c104a10 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.743+0000 7ff4214e5700 1 -- 172.21.15.178:0/4191251282 >> v1:172.21.15.178:6789/0 conn(0x7ff41c104a10 legacy=0x7ff41c104e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.743+0000 7ff4214e5700 1 -- 172.21.15.178:0/4191251282 shutdown_connections 2024-02-14T17:37:30.602 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.743+0000 7ff4214e5700 1 -- 172.21.15.178:0/4191251282 >> 172.21.15.178:0/4191251282 conn(0x7ff41c0fffd0 msgr2=0x7ff41c1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.743+0000 7ff4214e5700 1 -- 172.21.15.178:0/4191251282 shutdown_connections 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.743+0000 7ff4214e5700 1 -- 172.21.15.178:0/4191251282 wait complete. 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.744+0000 7ff4214e5700 1 Processor -- start 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.744+0000 7ff4214e5700 1 -- start start 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.744+0000 7ff4214e5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff41c199bb0 con 0x7ff41c104a10 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.744+0000 7ff41affd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff41c104a10 0x7ff41c1994a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:55214/0 (socket says 172.21.15.178:55214) 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.744+0000 7ff41affd700 1 -- 172.21.15.178:0/2247811535 learned_addr learned my addr 172.21.15.178:0/2247811535 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.744+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621201606 0 0) 0x7ff41c199bb0 con 0x7ff41c104a10 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.745+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4040054f0 con 0x7ff41c104a10 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.745+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3217953674 0 0) 0x7ff4040054f0 con 0x7ff41c104a10 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.745+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff41c199bb0 con 0x7ff41c104a10 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.745+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7ff40c001080 con 0x7ff41c104a10 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.745+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2715491066 0 0) 0x7ff41c199bb0 con 0x7ff41c104a10 2024-02-14T17:37:30.603 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.745+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c199d40 con 0x7ff41c104a10 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.745+0000 7ff4214e5700 1 -- 172.21.15.178:0/2247811535 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c199ff0 con 0x7ff41c104a10 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.745+0000 7ff4214e5700 1 -- 172.21.15.178:0/2247811535 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c19db40 con 0x7ff41c104a10 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.745+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7ff40c005940 con 0x7ff41c104a10 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.745+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7ff40c006360 con 0x7ff41c104a10 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.746+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7ff40c0119c0 con 0x7ff41c104a10 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.746+0000 7ff4214e5700 1 -- 172.21.15.178:0/2247811535 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff408005120 con 0x7ff41c104a10 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.746+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843057982 0 0) 0x7ff40c047710 con 0x7ff41c104a10 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:29.751+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7ff40c013770 con 0x7ff41c104a10 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:30.049+0000 7ff4214e5700 1 -- 172.21.15.178:0/2247811535 --> v1:172.21.15.178:6789/0 -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7ff408005950 con 0x7ff41c104a10 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:30.052+0000 7ff403fff700 1 -- 172.21.15.178:0/2247811535 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (unknown 38603050 0 0) 0x7ff40c016a80 con 0x7ff41c104a10 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:30.054+0000 7ff4214e5700 1 -- 172.21.15.178:0/2247811535 >> v1:172.21.15.178:6800/1500761607 conn(0x7ff40403a1c0 legacy=0x7ff40403c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:30.054+0000 7ff4214e5700 1 -- 172.21.15.178:0/2247811535 >> v1:172.21.15.178:6789/0 conn(0x7ff41c104a10 legacy=0x7ff41c1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:30.054+0000 7ff4214e5700 1 -- 172.21.15.178:0/2247811535 shutdown_connections 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:30.054+0000 7ff4214e5700 1 -- 172.21.15.178:0/2247811535 >> 172.21.15.178:0/2247811535 conn(0x7ff41c0fffd0 msgr2=0x7ff41c078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:30.604 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:30.054+0000 7ff4214e5700 1 -- 172.21.15.178:0/2247811535 shutdown_connections 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2024-02-14T17:37:30.054+0000 7ff4214e5700 1 -- 172.21.15.178:0/2247811535 wait complete. 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout: sudo /sbin/cephadm shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout:Or, if you are only running a single cluster on this host: 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout: sudo /sbin/cephadm shell 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout: ceph telemetry on 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:37:30.605 INFO:teuthology.orchestra.run.smithi178.stdout:For more information see: 2024-02-14T17:37:30.606 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:37:30.606 INFO:teuthology.orchestra.run.smithi178.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-14T17:37:30.606 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:37:30.606 INFO:teuthology.orchestra.run.smithi178.stdout:Bootstrap complete. 2024-02-14T17:37:30.639 INFO:tasks.cephadm:Fetching config... 2024-02-14T17:37:30.639 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:37:30.640 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-14T17:37:30.654 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-14T17:37:30.654 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:37:30.655 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-14T17:37:30.710 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-14T17:37:30.710 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:37:30.710 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-14T17:37:30.777 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-14T17:37:30.777 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:37:30.777 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-14T17:37:30.833 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-14T17:37:30.834 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKfRzz3lWgq+vxvk13JzmeQ3dAA7gF6mXX3bwTUSY35G+bcMkUV+HSAnc6YTF+UEVLkyxL48zbY2uryA25S8OngYLOsg7YvgdzK7mBjLPvNB8pLcvrdN9JLxFJf0NFxc70WWQlvpSGBeCmmP7wZ2gkSwx+z9KcAE4i9svOHsQXZ55A/g0KBkuABZBI9htEs4UzIZ584eXc1Q4ufnmcKM3WD/byY4Zr62orMAr7mOWP5gNNzM4JPfhPuWYJg7bNDkJarUexlOozlxgT7bYUW1jOb/S0ZnyLizBtd2Oo8vuF1NxCpyKnNamwspw7TTcJAud2TleNvGIvI+zbb+eRaNymmZ0r1ZdHlc797i+0AEosD11D1ljtqPH7q7Pi67/s7lvPoJO7ncLX3WcOdBj/OnFI/vbeLFGqMd9Ipo6XALz0B3EFrkfdTbRTGZUiK2uz2+eNtEsMhmk3sd3pietobWqre4DEleJPnplJEtQ+pvmDKKNAzzbl/QLnbDzhkp0oH5c= ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-14T17:37:30.913 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKfRzz3lWgq+vxvk13JzmeQ3dAA7gF6mXX3bwTUSY35G+bcMkUV+HSAnc6YTF+UEVLkyxL48zbY2uryA25S8OngYLOsg7YvgdzK7mBjLPvNB8pLcvrdN9JLxFJf0NFxc70WWQlvpSGBeCmmP7wZ2gkSwx+z9KcAE4i9svOHsQXZ55A/g0KBkuABZBI9htEs4UzIZ584eXc1Q4ufnmcKM3WD/byY4Zr62orMAr7mOWP5gNNzM4JPfhPuWYJg7bNDkJarUexlOozlxgT7bYUW1jOb/S0ZnyLizBtd2Oo8vuF1NxCpyKnNamwspw7TTcJAud2TleNvGIvI+zbb+eRaNymmZ0r1ZdHlc797i+0AEosD11D1ljtqPH7q7Pi67/s7lvPoJO7ncLX3WcOdBj/OnFI/vbeLFGqMd9Ipo6XALz0B3EFrkfdTbRTGZUiK2uz2+eNtEsMhmk3sd3pietobWqre4DEleJPnplJEtQ+pvmDKKNAzzbl/QLnbDzhkp0oH5c= ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:37:30.927 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKfRzz3lWgq+vxvk13JzmeQ3dAA7gF6mXX3bwTUSY35G+bcMkUV+HSAnc6YTF+UEVLkyxL48zbY2uryA25S8OngYLOsg7YvgdzK7mBjLPvNB8pLcvrdN9JLxFJf0NFxc70WWQlvpSGBeCmmP7wZ2gkSwx+z9KcAE4i9svOHsQXZ55A/g0KBkuABZBI9htEs4UzIZ584eXc1Q4ufnmcKM3WD/byY4Zr62orMAr7mOWP5gNNzM4JPfhPuWYJg7bNDkJarUexlOozlxgT7bYUW1jOb/S0ZnyLizBtd2Oo8vuF1NxCpyKnNamwspw7TTcJAud2TleNvGIvI+zbb+eRaNymmZ0r1ZdHlc797i+0AEosD11D1ljtqPH7q7Pi67/s7lvPoJO7ncLX3WcOdBj/OnFI/vbeLFGqMd9Ipo6XALz0B3EFrkfdTbRTGZUiK2uz2+eNtEsMhmk3sd3pietobWqre4DEleJPnplJEtQ+pvmDKKNAzzbl/QLnbDzhkp0oH5c= ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-14T17:37:30.975 INFO:teuthology.orchestra.run.smithi184.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKfRzz3lWgq+vxvk13JzmeQ3dAA7gF6mXX3bwTUSY35G+bcMkUV+HSAnc6YTF+UEVLkyxL48zbY2uryA25S8OngYLOsg7YvgdzK7mBjLPvNB8pLcvrdN9JLxFJf0NFxc70WWQlvpSGBeCmmP7wZ2gkSwx+z9KcAE4i9svOHsQXZ55A/g0KBkuABZBI9htEs4UzIZ584eXc1Q4ufnmcKM3WD/byY4Zr62orMAr7mOWP5gNNzM4JPfhPuWYJg7bNDkJarUexlOozlxgT7bYUW1jOb/S0ZnyLizBtd2Oo8vuF1NxCpyKnNamwspw7TTcJAud2TleNvGIvI+zbb+eRaNymmZ0r1ZdHlc797i+0AEosD11D1ljtqPH7q7Pi67/s7lvPoJO7ncLX3WcOdBj/OnFI/vbeLFGqMd9Ipo6XALz0B3EFrkfdTbRTGZUiK2uz2+eNtEsMhmk3sd3pietobWqre4DEleJPnplJEtQ+pvmDKKNAzzbl/QLnbDzhkp0oH5c= ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:37:30.990 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-14T17:37:31.189 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2247811535' entity='client.admin' 2024-02-14T17:37:31.264 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:37:33.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.513+0000 7f2bd5043700 1 -- 172.21.15.178:0/649855831 >> v1:172.21.15.178:6789/0 conn(0x7f2bd01015d0 legacy=0x7f2bd01019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.513+0000 7f2bd5043700 1 -- 172.21.15.178:0/649855831 shutdown_connections 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.513+0000 7f2bd5043700 1 -- 172.21.15.178:0/649855831 >> 172.21.15.178:0/649855831 conn(0x7f2bd00fcb90 msgr2=0x7f2bd00fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.513+0000 7f2bd5043700 1 -- 172.21.15.178:0/649855831 shutdown_connections 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.513+0000 7f2bd5043700 1 -- 172.21.15.178:0/649855831 wait complete. 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.514+0000 7f2bd5043700 1 Processor -- start 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.514+0000 7f2bd5043700 1 -- start start 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.514+0000 7f2bd5043700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2bd019be80 con 0x7f2bd01015d0 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.514+0000 7f2bced9d700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2bd01015d0 0x7f2bd0075930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:55090/0 (socket says 172.21.15.178:55090) 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.514+0000 7f2bced9d700 1 -- 172.21.15.178:0/2783697365 learned_addr learned my addr 172.21.15.178:0/2783697365 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.514+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451714817 0 0) 0x7f2bd019be80 con 0x7f2bd01015d0 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.515+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb80054f0 con 0x7f2bd01015d0 2024-02-14T17:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.515+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2597831653 0 0) 0x7f2bb80054f0 con 0x7f2bd01015d0 2024-02-14T17:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.515+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2bd019be80 con 0x7f2bd01015d0 2024-02-14T17:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.515+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2bc4004440 con 0x7f2bd01015d0 2024-02-14T17:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.515+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4133142929 0 0) 0x7f2bd019be80 con 0x7f2bd01015d0 2024-02-14T17:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.515+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd019d020 con 0x7f2bd01015d0 2024-02-14T17:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.515+0000 7f2bd5043700 1 -- 172.21.15.178:0/2783697365 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd019c070 con 0x7f2bd01015d0 2024-02-14T17:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.515+0000 7f2bd5043700 1 -- 172.21.15.178:0/2783697365 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd019c570 con 0x7f2bd01015d0 2024-02-14T17:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.515+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f2bc4001190 con 0x7f2bd01015d0 2024-02-14T17:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.515+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f2bc4004dc0 con 0x7f2bd01015d0 2024-02-14T17:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.516+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7f2bc4004f40 con 0x7f2bd01015d0 2024-02-14T17:37:33.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.516+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843057982 0 0) 0x7f2bc4048190 con 0x7f2bd01015d0 2024-02-14T17:37:33.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.516+0000 7f2bd5043700 1 -- 172.21.15.178:0/2783697365 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc005150 con 0x7f2bd01015d0 2024-02-14T17:37:33.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.521+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f2bc4016ac0 con 0x7f2bd01015d0 2024-02-14T17:37:33.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.716+0000 7f2bd5043700 1 -- 172.21.15.178:0/2783697365 --> v1:172.21.15.178:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f2bbc005980 con 0x7f2bd01015d0 2024-02-14T17:37:33.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.719+0000 7f2bb7fff700 1 -- 172.21.15.178:0/2783697365 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (unknown 3028693289 0 0) 0x7f2bc4014e90 con 0x7f2bd01015d0 2024-02-14T17:37:33.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.721+0000 7f2bd5043700 1 -- 172.21.15.178:0/2783697365 >> v1:172.21.15.178:6800/1500761607 conn(0x7f2bb803a2e0 legacy=0x7f2bb803c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:33.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.721+0000 7f2bd5043700 1 -- 172.21.15.178:0/2783697365 >> v1:172.21.15.178:6789/0 conn(0x7f2bd01015d0 legacy=0x7f2bd0075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:33.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.721+0000 7f2bd5043700 1 -- 172.21.15.178:0/2783697365 shutdown_connections 2024-02-14T17:37:33.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.721+0000 7f2bd5043700 1 -- 172.21.15.178:0/2783697365 >> 172.21.15.178:0/2783697365 conn(0x7f2bd00fcb90 msgr2=0x7f2bd00fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:33.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.721+0000 7f2bd5043700 1 -- 172.21.15.178:0/2783697365 shutdown_connections 2024-02-14T17:37:33.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:33.721+0000 7f2bd5043700 1 -- 172.21.15.178:0/2783697365 wait complete. 2024-02-14T17:37:33.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:33 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:33 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:33 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:37:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:33 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:33 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:37:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:33 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:34.243 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-14T17:37:34.243 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-14T17:37:34.478 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:37:34.842 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2783697365' entity='client.admin' 2024-02-14T17:37:34.842 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:34 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:37:34.842 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:34 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:35.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.959+0000 7f1b4bb39700 1 -- 172.21.15.178:0/2593877912 >> v1:172.21.15.178:6789/0 conn(0x7f1b440ffc80 legacy=0x7f1b441000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:35.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.959+0000 7f1b4bb39700 1 -- 172.21.15.178:0/2593877912 shutdown_connections 2024-02-14T17:37:35.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.959+0000 7f1b4bb39700 1 -- 172.21.15.178:0/2593877912 >> 172.21.15.178:0/2593877912 conn(0x7f1b440fb240 msgr2=0x7f1b440fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:35.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.959+0000 7f1b4bb39700 1 -- 172.21.15.178:0/2593877912 shutdown_connections 2024-02-14T17:37:35.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.959+0000 7f1b4bb39700 1 -- 172.21.15.178:0/2593877912 wait complete. 2024-02-14T17:37:35.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.960+0000 7f1b4bb39700 1 Processor -- start 2024-02-14T17:37:35.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.960+0000 7f1b4bb39700 1 -- start start 2024-02-14T17:37:35.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.960+0000 7f1b4bb39700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1b441965b0 con 0x7f1b440ffc80 2024-02-14T17:37:35.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.960+0000 7f1b498d5700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f1b440ffc80 0x7f1b44195ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:55108/0 (socket says 172.21.15.178:55108) 2024-02-14T17:37:35.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.960+0000 7f1b498d5700 1 -- 172.21.15.178:0/208864715 learned_addr learned my addr 172.21.15.178:0/208864715 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:35.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.961+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591940197 0 0) 0x7f1b441965b0 con 0x7f1b440ffc80 2024-02-14T17:37:35.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.961+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b300054f0 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.961+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2330848439 0 0) 0x7f1b300054f0 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.961+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1b441965b0 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.961+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f1b40004440 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.961+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3536467428 0 0) 0x7f1b441965b0 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.961+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b44196740 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.961+0000 7f1b4bb39700 1 -- 172.21.15.178:0/208864715 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b441969f0 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.961+0000 7f1b4bb39700 1 -- 172.21.15.178:0/208864715 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4419a4f0 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.961+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f1b40001620 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.961+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f1b40004eb0 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.962+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7f1b400050f0 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.962+0000 7f1b4bb39700 1 -- 172.21.15.178:0/208864715 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b28005150 con 0x7f1b440ffc80 2024-02-14T17:37:35.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.962+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843057982 0 0) 0x7f1b400475f0 con 0x7f1b440ffc80 2024-02-14T17:37:35.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:35.967+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f1b400134f0 con 0x7f1b440ffc80 2024-02-14T17:37:36.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:36.164+0000 7f1b4bb39700 1 -- 172.21.15.178:0/208864715 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b28000bc0 con 0x7f1b3003a1f0 2024-02-14T17:37:36.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:36.167+0000 7f1b3affd700 1 -- 172.21.15.178:0/208864715 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f1b28000bc0 con 0x7f1b3003a1f0 2024-02-14T17:37:36.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:36.169+0000 7f1b4bb39700 1 -- 172.21.15.178:0/208864715 >> v1:172.21.15.178:6800/1500761607 conn(0x7f1b3003a1f0 legacy=0x7f1b3003c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:36.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:36.169+0000 7f1b4bb39700 1 -- 172.21.15.178:0/208864715 >> v1:172.21.15.178:6789/0 conn(0x7f1b440ffc80 legacy=0x7f1b44195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:36.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:36.169+0000 7f1b4bb39700 1 -- 172.21.15.178:0/208864715 shutdown_connections 2024-02-14T17:37:36.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:36.169+0000 7f1b4bb39700 1 -- 172.21.15.178:0/208864715 >> 172.21.15.178:0/208864715 conn(0x7f1b440fb240 msgr2=0x7f1b44102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:36.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:36.169+0000 7f1b4bb39700 1 -- 172.21.15.178:0/208864715 shutdown_connections 2024-02-14T17:37:36.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:36.169+0000 7f1b4bb39700 1 -- 172.21.15.178:0/208864715 wait complete. 2024-02-14T17:37:36.900 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi184 2024-02-14T17:37:36.901 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:37:36.901 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.conf 2024-02-14T17:37:36.919 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:37:36.920 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:37:36.976 INFO:tasks.cephadm:Adding host smithi184 to orchestrator... 2024-02-14T17:37:36.976 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch host add smithi184 2024-02-14T17:37:37.226 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:37:37.432 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: from='client.14174 v1:172.21.15.178:0/208864715' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:37:37.432 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:37.432 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:37:37.432 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:37:37.432 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:37:37.432 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: Updating smithi178:/etc/ceph/ceph.conf 2024-02-14T17:37:37.432 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: Updating smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:37:37.433 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:37:37.433 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: Updating smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:37:37.433 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:37.433 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:37.433 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:37 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.690+0000 7f3d01b7a700 1 -- 172.21.15.178:0/1607544512 >> v1:172.21.15.178:6789/0 conn(0x7f3cfc102720 legacy=0x7f3cfc102b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.690+0000 7f3d01b7a700 1 -- 172.21.15.178:0/1607544512 shutdown_connections 2024-02-14T17:37:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.690+0000 7f3d01b7a700 1 -- 172.21.15.178:0/1607544512 >> 172.21.15.178:0/1607544512 conn(0x7f3cfc0fbf30 msgr2=0x7f3cfc0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.690+0000 7f3d01b7a700 1 -- 172.21.15.178:0/1607544512 shutdown_connections 2024-02-14T17:37:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.690+0000 7f3d01b7a700 1 -- 172.21.15.178:0/1607544512 wait complete. 2024-02-14T17:37:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.691+0000 7f3d01b7a700 1 Processor -- start 2024-02-14T17:37:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.691+0000 7f3d01b7a700 1 -- start start 2024-02-14T17:37:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.691+0000 7f3d01b7a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3cfc196930 con 0x7f3cfc102720 2024-02-14T17:37:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.691+0000 7f3cfb7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f3cfc102720 0x7f3cfc196220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:55124/0 (socket says 172.21.15.178:55124) 2024-02-14T17:37:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.691+0000 7f3cfb7fe700 1 -- 172.21.15.178:0/3004298541 learned_addr learned my addr 172.21.15.178:0/3004298541 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:38.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.692+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058903703 0 0) 0x7f3cfc196930 con 0x7f3cfc102720 2024-02-14T17:37:38.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.692+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ce40054f0 con 0x7f3cfc102720 2024-02-14T17:37:38.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.692+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4187606277 0 0) 0x7f3ce40054f0 con 0x7f3cfc102720 2024-02-14T17:37:38.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.692+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3cfc196930 con 0x7f3cfc102720 2024-02-14T17:37:38.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.692+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f3cec0044b0 con 0x7f3cfc102720 2024-02-14T17:37:38.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.692+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3403133695 0 0) 0x7f3cfc196930 con 0x7f3cfc102720 2024-02-14T17:37:38.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.692+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc196ac0 con 0x7f3cfc102720 2024-02-14T17:37:38.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.692+0000 7f3d01b7a700 1 -- 172.21.15.178:0/3004298541 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc196d70 con 0x7f3cfc102720 2024-02-14T17:37:38.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.692+0000 7f3d01b7a700 1 -- 172.21.15.178:0/3004298541 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc19ef50 con 0x7f3cfc102720 2024-02-14T17:37:38.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.692+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f3cec001620 con 0x7f3cfc102720 2024-02-14T17:37:38.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.692+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f3cec004dc0 con 0x7f3cfc102720 2024-02-14T17:37:38.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.693+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7f3cec004fe0 con 0x7f3cfc102720 2024-02-14T17:37:38.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.693+0000 7f3d01b7a700 1 -- 172.21.15.178:0/3004298541 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce8005120 con 0x7f3cfc102720 2024-02-14T17:37:38.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.693+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843057982 0 0) 0x7f3cec0471d0 con 0x7f3cfc102720 2024-02-14T17:37:38.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.698+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f3cec016880 con 0x7f3cfc102720 2024-02-14T17:37:38.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:38.896+0000 7f3d01b7a700 1 -- 172.21.15.178:0/3004298541 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ce8000b90 con 0x7f3ce403a290 2024-02-14T17:37:40.234 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:39 smithi178 ceph-mon[30321]: from='client.14176 v1:172.21.15.178:0/3004298541' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:37:40.234 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:39 smithi178 ceph-mon[30321]: Deploying cephadm binary to smithi184 2024-02-14T17:37:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:41 smithi178 ceph-mon[30321]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:43.164 INFO:teuthology.orchestra.run.smithi178.stdout:Added host 'smithi184' with addr '172.21.15.184' 2024-02-14T17:37:43.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:43.160+0000 7f3cf8ff9700 1 -- 172.21.15.178:0/3004298541 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 2438226024) 0x7f3ce8000b90 con 0x7f3ce403a290 2024-02-14T17:37:43.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:43.163+0000 7f3d01b7a700 1 -- 172.21.15.178:0/3004298541 >> v1:172.21.15.178:6800/1500761607 conn(0x7f3ce403a290 legacy=0x7f3ce403c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:43.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:43.163+0000 7f3d01b7a700 1 -- 172.21.15.178:0/3004298541 >> v1:172.21.15.178:6789/0 conn(0x7f3cfc102720 legacy=0x7f3cfc196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:43.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:43.163+0000 7f3d01b7a700 1 -- 172.21.15.178:0/3004298541 shutdown_connections 2024-02-14T17:37:43.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:43.163+0000 7f3d01b7a700 1 -- 172.21.15.178:0/3004298541 >> 172.21.15.178:0/3004298541 conn(0x7f3cfc0fbf30 msgr2=0x7f3cfc107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:43.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:43.163+0000 7f3d01b7a700 1 -- 172.21.15.178:0/3004298541 shutdown_connections 2024-02-14T17:37:43.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:43.163+0000 7f3d01b7a700 1 -- 172.21.15.178:0/3004298541 wait complete. 2024-02-14T17:37:43.783 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:43 smithi178 ceph-mon[30321]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:43.783 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:43.783 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:37:43.783 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:43.814 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch host ls --format=json 2024-02-14T17:37:44.055 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:37:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:44 smithi178 ceph-mon[30321]: Added host smithi184 2024-02-14T17:37:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:44 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:45.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.642+0000 7fd1f985a700 1 -- 172.21.15.178:0/4159907 >> v1:172.21.15.178:6789/0 conn(0x7fd1f41065b0 legacy=0x7fd1f4106990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:45.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.642+0000 7fd1f985a700 1 -- 172.21.15.178:0/4159907 shutdown_connections 2024-02-14T17:37:45.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.642+0000 7fd1f985a700 1 -- 172.21.15.178:0/4159907 >> 172.21.15.178:0/4159907 conn(0x7fd1f4075f60 msgr2=0x7fd1f4076370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:45.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.642+0000 7fd1f985a700 1 -- 172.21.15.178:0/4159907 shutdown_connections 2024-02-14T17:37:45.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.642+0000 7fd1f985a700 1 -- 172.21.15.178:0/4159907 wait complete. 2024-02-14T17:37:45.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.642+0000 7fd1f985a700 1 Processor -- start 2024-02-14T17:37:45.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.642+0000 7fd1f985a700 1 -- start start 2024-02-14T17:37:45.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f985a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd1f4196890 con 0x7fd1f41065b0 2024-02-14T17:37:45.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f2ffd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd1f41065b0 0x7fd1f4196180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:58924/0 (socket says 172.21.15.178:58924) 2024-02-14T17:37:45.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f2ffd700 1 -- 172.21.15.178:0/720275559 learned_addr learned my addr 172.21.15.178:0/720275559 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:45.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231945531 0 0) 0x7fd1f4196890 con 0x7fd1f41065b0 2024-02-14T17:37:45.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1e00054f0 con 0x7fd1f41065b0 2024-02-14T17:37:45.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3207658982 0 0) 0x7fd1e00054f0 con 0x7fd1f41065b0 2024-02-14T17:37:45.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd1f4196890 con 0x7fd1f41065b0 2024-02-14T17:37:45.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fd1e4004690 con 0x7fd1f41065b0 2024-02-14T17:37:45.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2826992005 0 0) 0x7fd1f4196890 con 0x7fd1f41065b0 2024-02-14T17:37:45.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f4196a20 con 0x7fd1f41065b0 2024-02-14T17:37:45.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f985a700 1 -- 172.21.15.178:0/720275559 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f4196cd0 con 0x7fd1f41065b0 2024-02-14T17:37:45.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f985a700 1 -- 172.21.15.178:0/720275559 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f419eee0 con 0x7fd1f41065b0 2024-02-14T17:37:45.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fd1e40011d0 con 0x7fd1f41065b0 2024-02-14T17:37:45.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.643+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7fd1e4004e40 con 0x7fd1f41065b0 2024-02-14T17:37:45.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.644+0000 7fd1f985a700 1 -- 172.21.15.178:0/720275559 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d4005120 con 0x7fd1f41065b0 2024-02-14T17:37:45.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.644+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7fd1e4005040 con 0x7fd1f41065b0 2024-02-14T17:37:45.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.645+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843057982 0 0) 0x7fd1e40473b0 con 0x7fd1f41065b0 2024-02-14T17:37:45.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.649+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fd1e4016bc0 con 0x7fd1f41065b0 2024-02-14T17:37:45.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.862+0000 7fd1f985a700 1 -- 172.21.15.178:0/720275559 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1d4000b90 con 0x7fd1e003a1f0 2024-02-14T17:37:45.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.864+0000 7fd1f8858700 1 -- 172.21.15.178:0/720275559 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (unknown 0 0 2869761869) 0x7fd1d4000b90 con 0x7fd1e003a1f0 2024-02-14T17:37:45.867 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:37:45.867 INFO:teuthology.orchestra.run.smithi178.stdout:[{"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}, {"addr": "172.21.15.184", "hostname": "smithi184", "labels": [], "status": ""}] 2024-02-14T17:37:45.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.866+0000 7fd1f985a700 1 -- 172.21.15.178:0/720275559 >> v1:172.21.15.178:6800/1500761607 conn(0x7fd1e003a1f0 legacy=0x7fd1e003c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.866+0000 7fd1f985a700 1 -- 172.21.15.178:0/720275559 >> v1:172.21.15.178:6789/0 conn(0x7fd1f41065b0 legacy=0x7fd1f4196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.866+0000 7fd1f985a700 1 -- 172.21.15.178:0/720275559 shutdown_connections 2024-02-14T17:37:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.866+0000 7fd1f985a700 1 -- 172.21.15.178:0/720275559 >> 172.21.15.178:0/720275559 conn(0x7fd1f4075f60 msgr2=0x7fd1f4069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.866+0000 7fd1f985a700 1 -- 172.21.15.178:0/720275559 shutdown_connections 2024-02-14T17:37:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:45.866+0000 7fd1f985a700 1 -- 172.21.15.178:0/720275559 wait complete. 2024-02-14T17:37:46.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:45 smithi178 ceph-mon[30321]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:46.439 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-14T17:37:46.440 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd crush tunables default 2024-02-14T17:37:46.672 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:37:47.014 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:46 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:47.014 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:46 smithi178 ceph-mon[30321]: from='client.14178 v1:172.21.15.178:0/720275559' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:37:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:47 smithi178 ceph-mon[30321]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:48.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.189+0000 7ff1fd65c700 1 -- 172.21.15.178:0/2978918636 >> v1:172.21.15.178:6789/0 conn(0x7ff1f8074010 legacy=0x7ff1f80743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.189+0000 7ff1fd65c700 1 -- 172.21.15.178:0/2978918636 shutdown_connections 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.189+0000 7ff1fd65c700 1 -- 172.21.15.178:0/2978918636 >> 172.21.15.178:0/2978918636 conn(0x7ff1f80fd070 msgr2=0x7ff1f80ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.189+0000 7ff1fd65c700 1 -- 172.21.15.178:0/2978918636 shutdown_connections 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.189+0000 7ff1fd65c700 1 -- 172.21.15.178:0/2978918636 wait complete. 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.189+0000 7ff1fd65c700 1 Processor -- start 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.189+0000 7ff1fd65c700 1 -- start start 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1fd65c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff1f81a2b70 con 0x7ff1f8074010 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1f6ffd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff1f8074010 0x7ff1f81a2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:58938/0 (socket says 172.21.15.178:58938) 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1f6ffd700 1 -- 172.21.15.178:0/399085664 learned_addr learned my addr 172.21.15.178:0/399085664 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979043652 0 0) 0x7ff1f81a2b70 con 0x7ff1f8074010 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e40054f0 con 0x7ff1f8074010 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3473202598 0 0) 0x7ff1e40054f0 con 0x7ff1f8074010 2024-02-14T17:37:48.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1f81a2b70 con 0x7ff1f8074010 2024-02-14T17:37:48.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7ff1e80044b0 con 0x7ff1f8074010 2024-02-14T17:37:48.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1042339800 0 0) 0x7ff1f81a2b70 con 0x7ff1f8074010 2024-02-14T17:37:48.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f81a2d00 con 0x7ff1f8074010 2024-02-14T17:37:48.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1fd65c700 1 -- 172.21.15.178:0/399085664 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f81a2fb0 con 0x7ff1f8074010 2024-02-14T17:37:48.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1fd65c700 1 -- 172.21.15.178:0/399085664 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f81a54e0 con 0x7ff1f8074010 2024-02-14T17:37:48.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7ff1e8001620 con 0x7ff1f8074010 2024-02-14T17:37:48.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.190+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7ff1e8004e30 con 0x7ff1f8074010 2024-02-14T17:37:48.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.191+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7ff1e8005070 con 0x7ff1f8074010 2024-02-14T17:37:48.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.191+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843057982 0 0) 0x7ff1e80473b0 con 0x7ff1f8074010 2024-02-14T17:37:48.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.192+0000 7ff1fd65c700 1 -- 172.21.15.178:0/399085664 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d8005120 con 0x7ff1f8074010 2024-02-14T17:37:48.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.197+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7ff1e8016b90 con 0x7ff1f8074010 2024-02-14T17:37:48.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.393+0000 7ff1fd65c700 1 -- 172.21.15.178:0/399085664 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff1d80053d0 con 0x7ff1f8074010 2024-02-14T17:37:48.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.753+0000 7ff1f4ff9700 1 -- 172.21.15.178:0/399085664 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (unknown 3126668360 0 0) 0x7ff1e8046ca0 con 0x7ff1f8074010 2024-02-14T17:37:48.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.755+0000 7ff1fd65c700 1 -- 172.21.15.178:0/399085664 >> v1:172.21.15.178:6800/1500761607 conn(0x7ff1e403a240 legacy=0x7ff1e403c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:48.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.755+0000 7ff1fd65c700 1 -- 172.21.15.178:0/399085664 >> v1:172.21.15.178:6789/0 conn(0x7ff1f8074010 legacy=0x7ff1f81a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:48.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.755+0000 7ff1fd65c700 1 -- 172.21.15.178:0/399085664 shutdown_connections 2024-02-14T17:37:48.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.755+0000 7ff1fd65c700 1 -- 172.21.15.178:0/399085664 >> 172.21.15.178:0/399085664 conn(0x7ff1f80fd070 msgr2=0x7ff1f80fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:48.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.755+0000 7ff1fd65c700 1 -- 172.21.15.178:0/399085664 shutdown_connections 2024-02-14T17:37:48.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:37:48.755+0000 7ff1fd65c700 1 -- 172.21.15.178:0/399085664 wait complete. 2024-02-14T17:37:48.758 INFO:teuthology.orchestra.run.smithi178.stderr:adjusted tunables profile to default 2024-02-14T17:37:49.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/399085664' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-14T17:37:49.467 INFO:tasks.cephadm:Adding mon.a on smithi178 2024-02-14T17:37:49.468 INFO:tasks.cephadm:Adding mon.c on smithi178 2024-02-14T17:37:49.468 INFO:tasks.cephadm:Adding mon.b on smithi184 2024-02-14T17:37:49.468 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply mon '3;smithi178:[v1:172.21.15.178:6789]=a;smithi178:[v1:172.21.15.178:6790]=c;smithi184:[v1:172.21.15.184:6789]=b' 2024-02-14T17:37:49.701 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/399085664' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-14T17:37:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:50.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:50.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:50 smithi178 ceph-mon[30321]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:37:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:50 smithi178 ceph-mon[30321]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:37:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:50 smithi178 ceph-mon[30321]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:37:51.133 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.130+0000 7f9309866700 1 -- 172.21.15.184:0/3508296706 >> v1:172.21.15.178:6789/0 conn(0x7f93041064e0 legacy=0x7f93041068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:51.133 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.130+0000 7f9309866700 1 -- 172.21.15.184:0/3508296706 shutdown_connections 2024-02-14T17:37:51.133 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.130+0000 7f9309866700 1 -- 172.21.15.184:0/3508296706 >> 172.21.15.184:0/3508296706 conn(0x7f93040fbf60 msgr2=0x7f93040fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:51.133 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.130+0000 7f9309866700 1 -- 172.21.15.184:0/3508296706 shutdown_connections 2024-02-14T17:37:51.133 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.130+0000 7f9309866700 1 -- 172.21.15.184:0/3508296706 wait complete. 2024-02-14T17:37:51.134 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.131+0000 7f9309866700 1 Processor -- start 2024-02-14T17:37:51.134 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.131+0000 7f9309866700 1 -- start start 2024-02-14T17:37:51.134 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.131+0000 7f9309866700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f93041968a0 con 0x7f93041064e0 2024-02-14T17:37:51.134 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9302ffd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f93041064e0 0x7f9304196190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.184:59946/0 (socket says 172.21.15.184:59946) 2024-02-14T17:37:51.134 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9302ffd700 1 -- 172.21.15.184:0/1869346878 learned_addr learned my addr 172.21.15.184:0/1869346878 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:37:51.134 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643330755 0 0) 0x7f93041968a0 con 0x7f93041064e0 2024-02-14T17:37:51.135 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92ec0054f0 con 0x7f93041064e0 2024-02-14T17:37:51.135 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1322515980 0 0) 0x7f92ec0054f0 con 0x7f93041064e0 2024-02-14T17:37:51.135 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f93041968a0 con 0x7f93041064e0 2024-02-14T17:37:51.135 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f92f4001980 con 0x7f93041064e0 2024-02-14T17:37:51.136 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 775362374 0 0) 0x7f93041968a0 con 0x7f93041064e0 2024-02-14T17:37:51.136 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9304196a30 con 0x7f93041064e0 2024-02-14T17:37:51.136 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9309866700 1 -- 172.21.15.184:0/1869346878 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9304196ce0 con 0x7f93041064e0 2024-02-14T17:37:51.136 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9309866700 1 -- 172.21.15.184:0/1869346878 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f930419eec0 con 0x7f93041064e0 2024-02-14T17:37:51.136 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f92f4001620 con 0x7f93041064e0 2024-02-14T17:37:51.136 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.132+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f92f4004e90 con 0x7f93041064e0 2024-02-14T17:37:51.136 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.133+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7f92f40050b0 con 0x7f93041064e0 2024-02-14T17:37:51.136 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.133+0000 7f9309866700 1 -- 172.21.15.184:0/1869346878 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f0005120 con 0x7f93041064e0 2024-02-14T17:37:51.136 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.133+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2087062522 0 0) 0x7f92f4047660 con 0x7f93041064e0 2024-02-14T17:37:51.141 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.139+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f92f4012ee0 con 0x7f93041064e0 2024-02-14T17:37:51.344 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.342+0000 7f9309866700 1 -- 172.21.15.184:0/1869346878 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi178:[v1:172.21.15.178:6789]=a;smithi178:[v1:172.21.15.178:6790]=c;smithi184:[v1:172.21.15.184:6789]=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f92f0000c70 con 0x7f92ec03a1f0 2024-02-14T17:37:51.351 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.349+0000 7f9308864700 1 -- 172.21.15.184:0/1869346878 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7f92f0000c70 con 0x7f92ec03a1f0 2024-02-14T17:37:51.351 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mon update... 2024-02-14T17:37:51.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.350+0000 7f9309866700 1 -- 172.21.15.184:0/1869346878 >> v1:172.21.15.178:6800/1500761607 conn(0x7f92ec03a1f0 legacy=0x7f92ec03c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:51.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.350+0000 7f9309866700 1 -- 172.21.15.184:0/1869346878 >> v1:172.21.15.178:6789/0 conn(0x7f93041064e0 legacy=0x7f9304196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:51.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.350+0000 7f9309866700 1 -- 172.21.15.184:0/1869346878 shutdown_connections 2024-02-14T17:37:51.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.350+0000 7f9309866700 1 -- 172.21.15.184:0/1869346878 >> 172.21.15.184:0/1869346878 conn(0x7f93040fbf60 msgr2=0x7f9304069a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:51.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.351+0000 7f9309866700 1 -- 172.21.15.184:0/1869346878 shutdown_connections 2024-02-14T17:37:51.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:51.351+0000 7f9309866700 1 -- 172.21.15.184:0/1869346878 wait complete. 2024-02-14T17:37:51.993 DEBUG:teuthology.orchestra.run.smithi178:mon.c> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.c.service 2024-02-14T17:37:51.996 DEBUG:teuthology.orchestra.run.smithi184:mon.b> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.b.service 2024-02-14T17:37:51.998 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T17:37:51.998 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T17:37:52.025 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:51 smithi178 ceph-mon[30321]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:52.025 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:52.025 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:37:52.025 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:37:52.025 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:37:52.025 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:52.025 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:37:52.025 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:37:52.773 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:37:52.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:52 smithi178 ceph-mon[30321]: from='client.14182 v1:172.21.15.184:0/1869346878' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi178:[v1:172.21.15.178:6789]=a;smithi178:[v1:172.21.15.178:6790]=c;smithi184:[v1:172.21.15.184:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:37:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:52 smithi178 ceph-mon[30321]: Saving service mon spec with placement smithi178:[v1:172.21.15.178:6789]=a;smithi178:[v1:172.21.15.178:6790]=c;smithi184:[v1:172.21.15.184:6789]=b;count:3 2024-02-14T17:37:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:52 smithi178 ceph-mon[30321]: Deploying daemon mon.b on smithi184 2024-02-14T17:37:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:53 smithi178 ceph-mon[30321]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:55.371 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.369+0000 7f8987cb1700 1 -- 172.21.15.184:0/183213615 >> v1:172.21.15.178:6789/0 conn(0x7f89801076d0 legacy=0x7f8980107ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:55.372 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.370+0000 7f8987cb1700 1 -- 172.21.15.184:0/183213615 shutdown_connections 2024-02-14T17:37:55.372 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.370+0000 7f8987cb1700 1 -- 172.21.15.184:0/183213615 >> 172.21.15.184:0/183213615 conn(0x7f89800fb700 msgr2=0x7f89800fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:55.372 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.370+0000 7f8987cb1700 1 -- 172.21.15.184:0/183213615 shutdown_connections 2024-02-14T17:37:55.372 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.370+0000 7f8987cb1700 1 -- 172.21.15.184:0/183213615 wait complete. 2024-02-14T17:37:55.372 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.370+0000 7f8987cb1700 1 Processor -- start 2024-02-14T17:37:55.372 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.370+0000 7f8987cb1700 1 -- start start 2024-02-14T17:37:55.374 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.371+0000 7f8987cb1700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f89801968c0 con 0x7f89801076d0 2024-02-14T17:37:55.374 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.371+0000 7f8985a4d700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f89801076d0 0x7f89801961b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.184:59964/0 (socket says 172.21.15.184:59964) 2024-02-14T17:37:55.374 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.371+0000 7f8985a4d700 1 -- 172.21.15.184:0/78465941 learned_addr learned my addr 172.21.15.184:0/78465941 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:37:55.374 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.371+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243971370 0 0) 0x7f89801968c0 con 0x7f89801076d0 2024-02-14T17:37:55.374 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.371+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f896c0054f0 con 0x7f89801076d0 2024-02-14T17:37:55.374 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.371+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1802181568 0 0) 0x7f896c0054f0 con 0x7f89801076d0 2024-02-14T17:37:55.374 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.371+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f89801968c0 con 0x7f89801076d0 2024-02-14T17:37:55.374 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.371+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f897c004440 con 0x7f89801076d0 2024-02-14T17:37:55.375 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.371+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1859452041 0 0) 0x7f89801968c0 con 0x7f89801076d0 2024-02-14T17:37:55.375 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.371+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8980196a50 con 0x7f89801076d0 2024-02-14T17:37:55.375 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.372+0000 7f8987cb1700 1 -- 172.21.15.184:0/78465941 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8980196d00 con 0x7f89801076d0 2024-02-14T17:37:55.375 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.372+0000 7f8987cb1700 1 -- 172.21.15.184:0/78465941 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f898019ef60 con 0x7f89801076d0 2024-02-14T17:37:55.375 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.372+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f897c001620 con 0x7f89801076d0 2024-02-14T17:37:55.375 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.372+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 2235730813 0 0) 0x7f897c004db0 con 0x7f89801076d0 2024-02-14T17:37:55.375 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.372+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7f897c004f90 con 0x7f89801076d0 2024-02-14T17:37:55.375 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.372+0000 7f8987cb1700 1 -- 172.21.15.184:0/78465941 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8964005120 con 0x7f89801076d0 2024-02-14T17:37:55.375 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.373+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2087062522 0 0) 0x7f897c047460 con 0x7f89801076d0 2024-02-14T17:37:55.379 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.378+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f897c016930 con 0x7f89801076d0 2024-02-14T17:37:55.656 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.654+0000 7f8987cb1700 1 -- 172.21.15.184:0/78465941 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f89640053d0 con 0x7f89801076d0 2024-02-14T17:37:55.656 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.655+0000 7f8976ffd700 1 -- 172.21.15.184:0/78465941 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+697 (unknown 2237029548 0 1610700781) 0x7f897c014f30 con 0x7f89801076d0 2024-02-14T17:37:55.657 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:37:55.657 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":1,"fsid":"6e1588ec-cb5f-11ee-95b9-87774f69a715","modified":"2024-02-14T17:36:17.375511Z","created":"2024-02-14T17:36:17.375511Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-14T17:37:55.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.657+0000 7f8987cb1700 1 -- 172.21.15.184:0/78465941 >> v1:172.21.15.178:6800/1500761607 conn(0x7f896c03a2b0 legacy=0x7f896c03c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:55.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.657+0000 7f8987cb1700 1 -- 172.21.15.184:0/78465941 >> v1:172.21.15.178:6789/0 conn(0x7f89801076d0 legacy=0x7f89801961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:37:55.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.657+0000 7f8987cb1700 1 -- 172.21.15.184:0/78465941 shutdown_connections 2024-02-14T17:37:55.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.657+0000 7f8987cb1700 1 -- 172.21.15.184:0/78465941 >> 172.21.15.184:0/78465941 conn(0x7f89800fb700 msgr2=0x7f8980108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:37:55.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.657+0000 7f8987cb1700 1 -- 172.21.15.184:0/78465941 shutdown_connections 2024-02-14T17:37:55.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:37:55.657+0000 7f8987cb1700 1 -- 172.21.15.184:0/78465941 wait complete. 2024-02-14T17:37:55.659 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 1 2024-02-14T17:37:56.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:55 smithi178 ceph-mon[30321]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:56.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:37:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/78465941' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:37:57.088 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:56 smithi184 podman[33499]: 2024-02-14 17:37:56.890245344 +0000 UTC m=+0.019202466 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:37:57.402 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T17:37:57.402 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T17:37:57.428 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 podman[33499]: 2024-02-14 17:37:57.144581524 +0000 UTC m=+0.273538633 container init 47a8f037abf289a0285c566e75c285c6423d09afebaf2ac946eea5e1d52f429e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-b, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-14T17:37:57.428 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 podman[33499]: 2024-02-14 17:37:57.150241568 +0000 UTC m=+0.279198685 container start 47a8f037abf289a0285c566e75c285c6423d09afebaf2ac946eea5e1d52f429e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-14T17:37:57.428 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: set uid:gid to 167:167 (ceph:ceph) 2024-02-14T17:37:57.428 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev), process ceph-mon, pid 7 2024-02-14T17:37:57.428 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: pidfile_write: ignore empty --pid-file 2024-02-14T17:37:57.428 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: load: jerasure load: lrc 2024-02-14T17:37:57.428 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: RocksDB version: 7.9.2 2024-02-14T17:37:57.428 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Git sha 0 2024-02-14T17:37:57.428 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Compile date 2024-02-14 11:59:27 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: DB SUMMARY 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: DB Session ID: FB8LFVK7COR0LSAUWPG9 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: CURRENT file: CURRENT 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: IDENTITY file: IDENTITY 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 476 ; 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.error_if_exists: 0 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.create_if_missing: 0 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.paranoid_checks: 1 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.env: 0x563edb3d8ae0 2024-02-14T17:37:57.429 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.fs: PosixFileSystem 2024-02-14T17:37:57.430 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.info_log: 0x563edc461920 2024-02-14T17:37:57.430 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_file_opening_threads: 16 2024-02-14T17:37:57.431 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.statistics: (nil) 2024-02-14T17:37:57.431 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.use_fsync: 0 2024-02-14T17:37:57.431 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_log_file_size: 0 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.keep_log_file_num: 1000 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.recycle_log_file_num: 0 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.allow_fallocate: 1 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.allow_mmap_reads: 0 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.allow_mmap_writes: 0 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.use_direct_reads: 0 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.create_missing_column_families: 0 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.db_log_dir: 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.wal_dir: 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-14T17:37:57.432 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.advise_random_on_open: 1 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.db_write_buffer_size: 0 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.write_buffer_manager: 0x563edc435400 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.rate_limiter: (nil) 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.wal_recovery_mode: 2 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.enable_thread_tracking: 0 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.enable_pipelined_write: 0 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.unordered_write: 0 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-14T17:37:57.433 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.row_cache: None 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.wal_filter: None 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.allow_ingest_behind: 0 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.two_write_queues: 0 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.manual_wal_flush: 0 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.wal_compression: 0 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.atomic_flush: 0 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.log_readahead_size: 0 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.best_efforts_recovery: 0 2024-02-14T17:37:57.434 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-14T17:37:57.435 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-14T17:37:57.435 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.allow_data_in_errors: 0 2024-02-14T17:37:57.435 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.db_host_id: __hostname__ 2024-02-14T17:37:57.435 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-14T17:37:57.435 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_background_jobs: 2 2024-02-14T17:37:57.435 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_background_compactions: -1 2024-02-14T17:37:57.435 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_subcompactions: 1 2024-02-14T17:37:57.435 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-14T17:37:57.435 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-14T17:37:57.435 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_total_wal_size: 0 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_open_files: -1 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bytes_per_sync: 0 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_readahead_size: 0 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_background_flushes: -1 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Compression algorithms supported: 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-14T17:37:57.436 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: kZSTD supported: 0 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: kXpressCompression supported: 0 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: kLZ4HCCompression supported: 1 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: kZlibCompression supported: 1 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: kSnappyCompression supported: 1 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: kLZ4Compression supported: 1 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: kBZip2Compression supported: 0 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.merge_operator: 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_filter: None 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_filter_factory: None 2024-02-14T17:37:57.437 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.sst_partitioner_factory: None 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563edc461a20) 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout: cache_index_and_filter_blocks: 1 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout: pin_top_level_index_and_filter: 1 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout: index_type: 0 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout: data_block_index_type: 0 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout: index_shortening: 1 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout: checksum: 4 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout: no_block_cache: 0 2024-02-14T17:37:57.438 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_cache: 0x563edc465090 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_cache_name: BinnedLRUCache 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_cache_options: 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: capacity : 536870912 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: num_shard_bits : 4 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: strict_capacity_limit : 0 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: high_pri_pool_ratio: 0.000 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_cache_compressed: (nil) 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: persistent_cache: (nil) 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_size: 4096 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_size_deviation: 10 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_restart_interval: 16 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: index_block_restart_interval: 1 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: metadata_block_size: 4096 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: partition_filters: 0 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: use_delta_encoding: 1 2024-02-14T17:37:57.439 INFO:journalctl@ceph.mon.b.smithi184.stdout: filter_policy: bloomfilter 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout: whole_key_filtering: 1 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout: verify_compression: 0 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout: read_amp_bytes_per_bit: 0 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout: format_version: 5 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout: enable_index_compression: 1 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_align: 0 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout: max_auto_readahead_size: 262144 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout: prepopulate_block_cache: 0 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout: initial_auto_readahead_size: 8192 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout: num_file_reads_for_auto_readahead: 2 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.write_buffer_size: 33554432 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_write_buffer_number: 2 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compression: NoCompression 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bottommost_compression: Disabled 2024-02-14T17:37:57.440 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.prefix_extractor: nullptr 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.num_levels: 7 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-14T17:37:57.441 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compression_opts.level: 32767 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compression_opts.strategy: 0 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compression_opts.enabled: false 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.target_file_size_base: 67108864 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-14T17:37:57.442 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.arena_block_size: 1048576 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.disable_auto_compactions: 0 2024-02-14T17:37:57.443 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.table_properties_collectors: 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.inplace_update_support: 0 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-14T17:37:57.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.bloom_locality: 0 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.max_successive_merges: 0 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.paranoid_file_checks: 0 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.force_consistency_checks: 1 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.report_bg_io_stats: 0 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.ttl: 2592000 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.enable_blob_files: false 2024-02-14T17:37:57.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.min_blob_size: 0 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.blob_file_size: 268435456 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.blob_file_starting_level: 0 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2250220a-861f-496e-8355-a583d48d5e0c 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932277180023, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932277180430, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1587, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 488, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 366, "raw_average_value_size": 73, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707932277, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2250220a-861f-496e-8355-a583d48d5e0c", "db_session_id": "FB8LFVK7COR0LSAUWPG9", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-14T17:37:57.446 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932277180516, "job": 1, "event": "recovery_finished"} 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563edc556000 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: DB pointer 0x563edc540000 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout: ** DB Stats ** 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:37:57.447 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-14T17:37:57.448 INFO:journalctl@ceph.mon.b.smithi184.stdout: ** Compaction Stats [default] ** 2024-02-14T17:37:57.448 INFO:journalctl@ceph.mon.b.smithi184.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:37:57.448 INFO:journalctl@ceph.mon.b.smithi184.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-14T17:37:57.448 INFO:journalctl@ceph.mon.b.smithi184.stdout: L0 1/0 1.55 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:37:57.448 INFO:journalctl@ceph.mon.b.smithi184.stdout: Sum 1/0 1.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:37:57.448 INFO:journalctl@ceph.mon.b.smithi184.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:37:57.448 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-14T17:37:57.448 INFO:journalctl@ceph.mon.b.smithi184.stdout: ** Compaction Stats [default] ** 2024-02-14T17:37:57.449 INFO:journalctl@ceph.mon.b.smithi184.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:37:57.449 INFO:journalctl@ceph.mon.b.smithi184.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-14T17:37:57.449 INFO:journalctl@ceph.mon.b.smithi184.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:37:57.449 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-14T17:37:57.449 INFO:journalctl@ceph.mon.b.smithi184.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-14T17:37:57.449 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-14T17:37:57.450 INFO:journalctl@ceph.mon.b.smithi184.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:37:57.450 INFO:journalctl@ceph.mon.b.smithi184.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-14T17:37:57.450 INFO:journalctl@ceph.mon.b.smithi184.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-14T17:37:57.450 INFO:journalctl@ceph.mon.b.smithi184.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-14T17:37:57.450 INFO:journalctl@ceph.mon.b.smithi184.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-14T17:37:57.450 INFO:journalctl@ceph.mon.b.smithi184.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-14T17:37:57.450 INFO:journalctl@ceph.mon.b.smithi184.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:37:57.451 INFO:journalctl@ceph.mon.b.smithi184.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:37:57.451 INFO:journalctl@ceph.mon.b.smithi184.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-14T17:37:57.451 INFO:journalctl@ceph.mon.b.smithi184.stdout: Block cache BinnedLRUCache@0x563edc465090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-14T17:37:57.451 INFO:journalctl@ceph.mon.b.smithi184.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-14T17:37:57.451 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-14T17:37:57.451 INFO:journalctl@ceph.mon.b.smithi184.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-14T17:37:57.452 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-14T17:37:57.452 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: using public_addrv v1:172.21.15.184:6789/0 2024-02-14T17:37:57.452 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: starting mon.b rank -1 at public addrs v1:172.21.15.184:6789/0 at bind addrs v1:172.21.15.184:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:37:57.452 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(???) e0 preinit fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:37:57.452 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).mds e1 new map 2024-02-14T17:37:57.452 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-14T17:37:57.452 INFO:journalctl@ceph.mon.b.smithi184.stdout: e1 2024-02-14T17:37:57.453 INFO:journalctl@ceph.mon.b.smithi184.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T17:37:57.453 INFO:journalctl@ceph.mon.b.smithi184.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T17:37:57.453 INFO:journalctl@ceph.mon.b.smithi184.stdout: legacy client fscid: -1 2024-02-14T17:37:57.453 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-14T17:37:57.453 INFO:journalctl@ceph.mon.b.smithi184.stdout: No filesystems configured 2024-02-14T17:37:57.453 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-14T17:37:57.453 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-14T17:37:57.454 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-14T17:37:57.454 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-14T17:37:57.454 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-14T17:37:57.454 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-14T17:37:57.454 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-14T17:37:57.454 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:37:57.454 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:37:57.455 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:37:57.455 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:57.455 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/399085664' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-14T17:37:57.455 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:57.455 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/399085664' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-14T17:37:57.455 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:37:57.455 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:57.456 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:57.456 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:57.456 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:57.456 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:37:57.456 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:37:57.456 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:37:57.457 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-14T17:37:57.457 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:57.457 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:57.457 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:57.457 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:37:57.457 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:37:57.457 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:37:57.458 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:57.458 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:57.458 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:37:57.458 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:37:57.458 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:37:57.458 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:37:57.459 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:37:57.459 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:37:57.459 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='client.14182 v1:172.21.15.184:0/1869346878' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi178:[v1:172.21.15.178:6789]=a;smithi178:[v1:172.21.15.178:6790]=c;smithi184:[v1:172.21.15.184:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:37:57.459 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: Saving service mon spec with placement smithi178:[v1:172.21.15.178:6789]=a;smithi178:[v1:172.21.15.178:6790]=c;smithi184:[v1:172.21.15.184:6789]=b;count:3 2024-02-14T17:37:57.459 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: Deploying daemon mon.b on smithi184 2024-02-14T17:37:57.459 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:57.459 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:37:57.460 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/78465941' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:37:57.460 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 ceph-mon[33536]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-14T17:37:57.460 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 bash[33499]: 47a8f037abf289a0285c566e75c285c6423d09afebaf2ac946eea5e1d52f429e 2024-02-14T17:37:57.460 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:37:57 smithi184 systemd[1]: Started Ceph mon.b for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:37:57.636 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:38:02.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.254+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/2888225653 >> v1:172.21.15.178:6789/0 conn(0x7fcc280035f0 legacy=0x7fcc28005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:02.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.254+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/2888225653 shutdown_connections 2024-02-14T17:38:02.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.254+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/2888225653 >> 172.21.15.184:0/2888225653 conn(0x7fcc440fc140 msgr2=0x7fcc440fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:38:02.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.255+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/2888225653 shutdown_connections 2024-02-14T17:38:02.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.255+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/2888225653 wait complete. 2024-02-14T17:38:02.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.255+0000 7fcc4b0b0700 1 Processor -- start 2024-02-14T17:38:02.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.255+0000 7fcc4b0b0700 1 -- start start 2024-02-14T17:38:02.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.256+0000 7fcc4b0b0700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcc441a45a0 con 0x7fcc44198a20 2024-02-14T17:38:02.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.256+0000 7fcc4b0b0700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcc441a5740 con 0x7fcc280035f0 2024-02-14T17:38:02.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.256+0000 7fcc48e4c700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fcc280035f0 0x7fcc44198310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:54340/0 (socket says 172.21.15.184:54340) 2024-02-14T17:38:02.260 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.256+0000 7fcc43fff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fcc44198a20 0x7fcc441a2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.184:45530/0 (socket says 172.21.15.184:45530) 2024-02-14T17:38:02.260 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.256+0000 7fcc48e4c700 1 -- 172.21.15.184:0/3807371272 learned_addr learned my addr 172.21.15.184:0/3807371272 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:38:02.260 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.256+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663681082 0 0) 0x7fcc441a45a0 con 0x7fcc44198a20 2024-02-14T17:38:02.260 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.256+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc340035b0 con 0x7fcc44198a20 2024-02-14T17:38:02.260 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.257+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 901589586 0 0) 0x7fcc340035b0 con 0x7fcc44198a20 2024-02-14T17:38:02.260 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.257+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcc441a45a0 con 0x7fcc44198a20 2024-02-14T17:38:02.260 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.257+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 3902029764 0 0) 0x7fcc300017e0 con 0x7fcc44198a20 2024-02-14T17:38:02.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.257+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2552558815 0 0) 0x7fcc441a45a0 con 0x7fcc44198a20 2024-02-14T17:38:02.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.257+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 >> v1:172.21.15.184:6789/0 conn(0x7fcc280035f0 legacy=0x7fcc44198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:02.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.257+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc441a68e0 con 0x7fcc44198a20 2024-02-14T17:38:02.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.257+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/3807371272 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc441a58d0 con 0x7fcc44198a20 2024-02-14T17:38:02.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.257+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fcc30004d00 con 0x7fcc44198a20 2024-02-14T17:38:02.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.257+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 3902029764 0 0) 0x7fcc30005c90 con 0x7fcc44198a20 2024-02-14T17:38:02.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.258+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/3807371272 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc441a5e10 con 0x7fcc44198a20 2024-02-14T17:38:02.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.259+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/3807371272 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc440632a0 con 0x7fcc44198a20 2024-02-14T17:38:02.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.261+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7fcc30004870 con 0x7fcc44198a20 2024-02-14T17:38:02.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.261+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2087062522 0 0) 0x7fcc300470f0 con 0x7fcc44198a20 2024-02-14T17:38:02.267 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.265+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fcc30015a60 con 0x7fcc44198a20 2024-02-14T17:38:02.551 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.549+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/3807371272 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcc44199670 con 0x7fcc44198a20 2024-02-14T17:38:02.551 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.550+0000 7fcc41ffb700 1 -- 172.21.15.184:0/3807371272 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+915 (unknown 3557084514 0 3061254130) 0x7fcc30015850 con 0x7fcc44198a20 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: mon.a calling monitor election 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: mon.b calling monitor election 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-14T17:38:02.552 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: monmap e2: 2 mons at {a=v1:172.21.15.178:6789/0,b=v1:172.21.15.184:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:38:02.553 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: fsmap 2024-02-14T17:38:02.553 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:38:02.553 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: mgrmap e13: y(active, since 41s) 2024-02-14T17:38:02.553 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: overall HEALTH_OK 2024-02-14T17:38:02.553 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:02.553 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:02.553 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:02.553 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:02.553 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:02.553 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:38:02.553 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"6e1588ec-cb5f-11ee-95b9-87774f69a715","modified":"2024-02-14T17:37:57.205605Z","created":"2024-02-14T17:36:17.375511Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-14T17:38:02.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.552+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/3807371272 >> v1:172.21.15.178:6800/1500761607 conn(0x7fcc3403a350 legacy=0x7fcc3403c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:02.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.552+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/3807371272 >> v1:172.21.15.178:6789/0 conn(0x7fcc44198a20 legacy=0x7fcc441a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:02.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.552+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/3807371272 shutdown_connections 2024-02-14T17:38:02.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.552+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/3807371272 >> 172.21.15.184:0/3807371272 conn(0x7fcc440fc140 msgr2=0x7fcc4406a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:38:02.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.552+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/3807371272 shutdown_connections 2024-02-14T17:38:02.554 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:02.552+0000 7fcc4b0b0700 1 -- 172.21.15.184:0/3807371272 wait complete. 2024-02-14T17:38:02.555 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: mon.a calling monitor election 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: mon.b calling monitor election 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:02.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-14T17:38:02.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: monmap e2: 2 mons at {a=v1:172.21.15.178:6789/0,b=v1:172.21.15.184:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:38:02.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: fsmap 2024-02-14T17:38:02.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:38:02.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: mgrmap e13: y(active, since 41s) 2024-02-14T17:38:02.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: overall HEALTH_OK 2024-02-14T17:38:02.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:02.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:02.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:02.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:02.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:03.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:03 smithi184 ceph-mon[33536]: Deploying daemon mon.c on smithi178 2024-02-14T17:38:03.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/3807371272' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:38:03.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:03 smithi184 ceph-mon[33536]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:03.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:03 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:03.578 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:03 smithi178 ceph-mon[30321]: Deploying daemon mon.c on smithi178 2024-02-14T17:38:03.578 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/3807371272' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:38:03.579 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:03 smithi178 ceph-mon[30321]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:03.579 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:03 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:04.163 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T17:38:04.163 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T17:38:04.407 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:38:05.487 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 podman[39772]: 2024-02-14 17:38:05.369470853 +0000 UTC m=+0.131539730 container create ede2979ba29a19db2de016e067a6ce06720446cea0a8ef6de9cd463f09155394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:38:06.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 podman[39772]: 2024-02-14 17:38:05.59719084 +0000 UTC m=+0.359259717 container init ede2979ba29a19db2de016e067a6ce06720446cea0a8ef6de9cd463f09155394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T17:38:06.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 podman[39772]: 2024-02-14 17:38:05.603459842 +0000 UTC m=+0.365528720 container start ede2979ba29a19db2de016e067a6ce06720446cea0a8ef6de9cd463f09155394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: set uid:gid to 167:167 (ceph:ceph) 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev), process ceph-mon, pid 7 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: pidfile_write: ignore empty --pid-file 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: load: jerasure load: lrc 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: RocksDB version: 7.9.2 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Git sha 0 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Compile date 2024-02-14 11:59:27 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: DB SUMMARY 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: DB Session ID: ZTOIVASP69Q05MKFD5CN 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: CURRENT file: CURRENT 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: IDENTITY file: IDENTITY 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-14T17:38:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-14T17:38:06.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 566 ; 2024-02-14T17:38:06.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.error_if_exists: 0 2024-02-14T17:38:06.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.create_if_missing: 0 2024-02-14T17:38:06.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.paranoid_checks: 1 2024-02-14T17:38:06.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-14T17:38:06.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-14T17:38:06.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-14T17:38:06.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.env: 0x55eb0914bae0 2024-02-14T17:38:06.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.fs: PosixFileSystem 2024-02-14T17:38:06.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.info_log: 0x55eb09e5b8e0 2024-02-14T17:38:06.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_file_opening_threads: 16 2024-02-14T17:38:06.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.statistics: (nil) 2024-02-14T17:38:06.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.use_fsync: 0 2024-02-14T17:38:06.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_log_file_size: 0 2024-02-14T17:38:06.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-14T17:38:06.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-14T17:38:06.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.keep_log_file_num: 1000 2024-02-14T17:38:06.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.recycle_log_file_num: 0 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.allow_fallocate: 1 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.allow_mmap_reads: 0 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.allow_mmap_writes: 0 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.use_direct_reads: 0 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.create_missing_column_families: 0 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.db_log_dir: 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.wal_dir: 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-14T17:38:06.088 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.advise_random_on_open: 1 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.db_write_buffer_size: 0 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.write_buffer_manager: 0x55eb09e2f400 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.rate_limiter: (nil) 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.wal_recovery_mode: 2 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.enable_thread_tracking: 0 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.enable_pipelined_write: 0 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.unordered_write: 0 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-14T17:38:06.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.row_cache: None 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.wal_filter: None 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.allow_ingest_behind: 0 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.two_write_queues: 0 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.manual_wal_flush: 0 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.wal_compression: 0 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.atomic_flush: 0 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.log_readahead_size: 0 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.best_efforts_recovery: 0 2024-02-14T17:38:06.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.allow_data_in_errors: 0 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.db_host_id: __hostname__ 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_background_jobs: 2 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_background_compactions: -1 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_subcompactions: 1 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_total_wal_size: 0 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-14T17:38:06.091 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_open_files: -1 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bytes_per_sync: 0 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_readahead_size: 0 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_background_flushes: -1 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Compression algorithms supported: 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: kZSTD supported: 0 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: kXpressCompression supported: 0 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: kLZ4HCCompression supported: 1 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: kZlibCompression supported: 1 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: kSnappyCompression supported: 1 2024-02-14T17:38:06.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: kLZ4Compression supported: 1 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: kBZip2Compression supported: 0 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.merge_operator: 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_filter: None 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_filter_factory: None 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.sst_partitioner_factory: None 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55eb09e5b9e0) 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks: 1 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-14T17:38:06.093 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_top_level_index_and_filter: 1 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_type: 0 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_index_type: 0 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_shortening: 1 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: checksum: 4 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: no_block_cache: 0 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache: 0x55eb09e5f090 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_name: BinnedLRUCache 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_options: 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: capacity : 536870912 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_shard_bits : 4 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: strict_capacity_limit : 0 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: high_pri_pool_ratio: 0.000 2024-02-14T17:38:06.094 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_compressed: (nil) 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: persistent_cache: (nil) 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size: 4096 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size_deviation: 10 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_restart_interval: 16 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_block_restart_interval: 1 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: metadata_block_size: 4096 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: partition_filters: 0 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: use_delta_encoding: 1 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: filter_policy: bloomfilter 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: whole_key_filtering: 1 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: verify_compression: 0 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: read_amp_bytes_per_bit: 0 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: format_version: 5 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_index_compression: 1 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_align: 0 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: max_auto_readahead_size: 262144 2024-02-14T17:38:06.095 INFO:journalctl@ceph.mon.c.smithi178.stdout: prepopulate_block_cache: 0 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout: initial_auto_readahead_size: 8192 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.write_buffer_size: 33554432 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_write_buffer_number: 2 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compression: NoCompression 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bottommost_compression: Disabled 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.prefix_extractor: nullptr 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.num_levels: 7 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-14T17:38:06.096 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compression_opts.level: 32767 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compression_opts.strategy: 0 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:38:06.097 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compression_opts.enabled: false 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.target_file_size_base: 67108864 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-14T17:38:06.098 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.arena_block_size: 1048576 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.disable_auto_compactions: 0 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-14T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.table_properties_collectors: 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.inplace_update_support: 0 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-14T17:38:06.100 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.bloom_locality: 0 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.max_successive_merges: 0 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.paranoid_file_checks: 0 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.force_consistency_checks: 1 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.report_bg_io_stats: 0 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.ttl: 2592000 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.enable_blob_files: false 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.min_blob_size: 0 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.blob_file_size: 268435456 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-14T17:38:06.101 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.blob_file_starting_level: 0 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 146eb0d5-1aad-41f8-a792-17b8842904e7 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932285633188, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932285633572, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1677, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 578, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 456, "raw_average_value_size": 91, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707932285, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "146eb0d5-1aad-41f8-a792-17b8842904e7", "db_session_id": "ZTOIVASP69Q05MKFD5CN", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932285633652, "job": 1, "event": "recovery_finished"} 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55eb09f50000 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: DB pointer 0x55eb09f3a000 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-14T17:38:06.102 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** DB Stats ** 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: L0 1/0 1.64 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: Sum 1/0 1.64 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:38:06.103 INFO:journalctl@ceph.mon.c.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-14T17:38:06.104 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache BinnedLRUCache@0x55eb09e5f090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: using public_addrv v1:172.21.15.178:6790/0 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: starting mon.c rank -1 at public addrs v1:172.21.15.178:6790/0 at bind addrs v1:172.21.15.178:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(???) e0 preinit fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).mds e1 new map 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout: e1 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T17:38:06.105 INFO:journalctl@ceph.mon.c.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout: legacy client fscid: -1 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout: No filesystems configured 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:06.106 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/399085664' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/399085664' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.107 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='client.14182 v1:172.21.15.184:0/1869346878' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi178:[v1:172.21.15.178:6789]=a;smithi178:[v1:172.21.15.178:6790]=c;smithi184:[v1:172.21.15.184:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: Saving service mon spec with placement smithi178:[v1:172.21.15.178:6789]=a;smithi178:[v1:172.21.15.178:6790]=c;smithi184:[v1:172.21.15.184:6789]=b;count:3 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: Deploying daemon mon.b on smithi184 2024-02-14T17:38:06.108 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:06.109 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:06.109 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/78465941' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:38:06.109 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:38:06.109 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.a calling monitor election 2024-02-14T17:38:06.109 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:06.109 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:06.109 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:06.109 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:06.109 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.b calling monitor election 2024-02-14T17:38:06.109 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: monmap e2: 2 mons at {a=v1:172.21.15.178:6789/0,b=v1:172.21.15.184:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: fsmap 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mgrmap e13: y(active, since 41s) 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: overall HEALTH_OK 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:06.110 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:06.111 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: Deploying daemon mon.c on smithi178 2024-02-14T17:38:06.111 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/3807371272' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:38:06.111 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:06.111 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:06.111 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 ceph-mon[39811]: mon.c@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-02-14T17:38:06.111 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 bash[39772]: ede2979ba29a19db2de016e067a6ce06720446cea0a8ef6de9cd463f09155394 2024-02-14T17:38:06.111 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:05 smithi178 systemd[1]: Started Ceph mon.c for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:38:10.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.703+0000 7f4e83ebd700 1 -- 172.21.15.184:0/4020743275 >> v1:172.21.15.178:6789/0 conn(0x7f4e600035f0 legacy=0x7f4e60005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:10.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.703+0000 7f4e83ebd700 1 -- 172.21.15.184:0/4020743275 shutdown_connections 2024-02-14T17:38:10.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.703+0000 7f4e83ebd700 1 -- 172.21.15.184:0/4020743275 >> 172.21.15.184:0/4020743275 conn(0x7f4e7c0fc0a0 msgr2=0x7f4e7c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:38:10.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.703+0000 7f4e83ebd700 1 -- 172.21.15.184:0/4020743275 shutdown_connections 2024-02-14T17:38:10.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.703+0000 7f4e83ebd700 1 -- 172.21.15.184:0/4020743275 wait complete. 2024-02-14T17:38:10.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.704+0000 7f4e83ebd700 1 Processor -- start 2024-02-14T17:38:10.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.704+0000 7f4e83ebd700 1 -- start start 2024-02-14T17:38:10.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.704+0000 7f4e83ebd700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4e7c1a6150 con 0x7f4e600035f0 2024-02-14T17:38:10.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.704+0000 7f4e83ebd700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4e7c1a7330 con 0x7f4e7c196900 2024-02-14T17:38:10.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.704+0000 7f4e83ebd700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4e7c1a8530 con 0x7f4e7c1a2480 2024-02-14T17:38:10.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.704+0000 7f4e81458700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f4e7c196900 0x7f4e7c1a0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:33904/0 (socket says 172.21.15.184:33904) 2024-02-14T17:38:10.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.704+0000 7f4e81c59700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f4e600035f0 0x7f4e7c1961f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.184:58560/0 (socket says 172.21.15.184:58560) 2024-02-14T17:38:10.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.704+0000 7f4e81458700 1 -- 172.21.15.184:0/3608208330 learned_addr learned my addr 172.21.15.184:0/3608208330 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:38:10.708 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.705+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509133523 0 0) 0x7f4e7c1a6150 con 0x7f4e600035f0 2024-02-14T17:38:10.708 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.705+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e68003bb0 con 0x7f4e600035f0 2024-02-14T17:38:10.708 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312067189 0 0) 0x7f4e7c1a7330 con 0x7f4e7c196900 2024-02-14T17:38:10.708 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e7c1a6150 con 0x7f4e7c196900 2024-02-14T17:38:10.708 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1693453899 0 0) 0x7f4e68003bb0 con 0x7f4e600035f0 2024-02-14T17:38:10.708 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4e7c1a7330 con 0x7f4e600035f0 2024-02-14T17:38:10.708 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f4e6c001120 con 0x7f4e600035f0 2024-02-14T17:38:10.709 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2094420839 0 0) 0x7f4e7c1a7330 con 0x7f4e600035f0 2024-02-14T17:38:10.709 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 >> v1:172.21.15.178:6790/0 conn(0x7f4e7c1a2480 legacy=0x7f4e7c1a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:10.709 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 >> v1:172.21.15.184:6789/0 conn(0x7f4e7c196900 legacy=0x7f4e7c1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:10.709 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c1a9730 con 0x7f4e600035f0 2024-02-14T17:38:10.709 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e83ebd700 1 -- 172.21.15.184:0/3608208330 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7c1a6340 con 0x7f4e600035f0 2024-02-14T17:38:10.709 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e83ebd700 1 -- 172.21.15.184:0/3608208330 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7c1a6880 con 0x7f4e600035f0 2024-02-14T17:38:10.709 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f4e6c0045a0 con 0x7f4e600035f0 2024-02-14T17:38:10.709 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.706+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f4e6c004e80 con 0x7f4e600035f0 2024-02-14T17:38:10.710 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.707+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7f4e6c005000 con 0x7f4e600035f0 2024-02-14T17:38:10.710 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.708+0000 7f4e83ebd700 1 -- 172.21.15.184:0/3608208330 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e60006b90 con 0x7f4e600035f0 2024-02-14T17:38:10.710 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.708+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2087062522 0 0) 0x7f4e6c047190 con 0x7f4e600035f0 2024-02-14T17:38:10.716 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:10.715+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f4e6c005440 con 0x7f4e600035f0 2024-02-14T17:38:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:38:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: mon.a calling monitor election 2024-02-14T17:38:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: mon.b calling monitor election 2024-02-14T17:38:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: mon.c calling monitor election 2024-02-14T17:38:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: monmap e3: 3 mons at {a=v1:172.21.15.178:6789/0,b=v1:172.21.15.184:6789/0,c=v1:172.21.15.178:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: fsmap 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: mgrmap e13: y(active, since 49s) 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: overall HEALTH_OK 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:10.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.002 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:11.001+0000 7f4e83ebd700 1 -- 172.21.15.184:0/3608208330 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4e60006e40 con 0x7f4e600035f0 2024-02-14T17:38:11.003 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:11.002+0000 7f4e72ffd700 1 -- 172.21.15.184:0/3608208330 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1133 (unknown 409899127 0 2801574629) 0x7f4e6c014db0 con 0x7f4e600035f0 2024-02-14T17:38:11.003 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-14T17:38:11.003 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":3,"fsid":"6e1588ec-cb5f-11ee-95b9-87774f69a715","modified":"2024-02-14T17:38:05.652305Z","created":"2024-02-14T17:36:17.375511Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-14T17:38:11.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:11.004+0000 7f4e83ebd700 1 -- 172.21.15.184:0/3608208330 >> v1:172.21.15.178:6800/1500761607 conn(0x7f4e6803a2a0 legacy=0x7f4e6803c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:11.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:11.004+0000 7f4e83ebd700 1 -- 172.21.15.184:0/3608208330 >> v1:172.21.15.178:6789/0 conn(0x7f4e600035f0 legacy=0x7f4e7c1961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:11.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:11.004+0000 7f4e83ebd700 1 -- 172.21.15.184:0/3608208330 shutdown_connections 2024-02-14T17:38:11.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:11.004+0000 7f4e83ebd700 1 -- 172.21.15.184:0/3608208330 >> 172.21.15.184:0/3608208330 conn(0x7f4e7c0fc0a0 msgr2=0x7f4e7c069590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:38:11.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:11.004+0000 7f4e83ebd700 1 -- 172.21.15.184:0/3608208330 shutdown_connections 2024-02-14T17:38:11.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:11.004+0000 7f4e83ebd700 1 -- 172.21.15.184:0/3608208330 wait complete. 2024-02-14T17:38:11.007 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 3 2024-02-14T17:38:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:38:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: mon.a calling monitor election 2024-02-14T17:38:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: mon.b calling monitor election 2024-02-14T17:38:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: mon.c calling monitor election 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: monmap e3: 3 mons at {a=v1:172.21.15.178:6789/0,b=v1:172.21.15.184:6789/0,c=v1:172.21.15.178:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: fsmap 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: mgrmap e13: y(active, since 49s) 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: overall HEALTH_OK 2024-02-14T17:38:11.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: mon.a calling monitor election 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: mon.b calling monitor election 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: mon.c calling monitor election 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: monmap e3: 3 mons at {a=v1:172.21.15.178:6789/0,b=v1:172.21.15.184:6789/0,c=v1:172.21.15.178:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: fsmap 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: mgrmap e13: y(active, since 49s) 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: overall HEALTH_OK 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.622 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-14T17:38:11.622 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph config generate-minimal-conf 2024-02-14T17:38:11.884 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:11 smithi178 ceph-mon[30321]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:11.884 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:11 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.884 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:11 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:11.884 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/3608208330' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:38:11.884 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:11 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.884 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:11 smithi178 ceph-mon[39811]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:11.884 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:11 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.884 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:11 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:11.884 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/3608208330' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:38:11.884 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:11 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:11.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:11 smithi184 ceph-mon[33536]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:11.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:11 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:11 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/3608208330' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:38:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:11 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:38:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:38:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: Updating smithi178:/etc/ceph/ceph.conf 2024-02-14T17:38:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-14T17:38:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: Updating smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:38:13.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: Updating smithi178:/etc/ceph/ceph.conf 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: Updating smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:13.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:38:13.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:13.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: Updating smithi178:/etc/ceph/ceph.conf 2024-02-14T17:38:13.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-14T17:38:13.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:13.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: Updating smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:38:13.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:38:13.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:13.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:13.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:38:13.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:13.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.949+0000 7f5796ed6700 1 -- 172.21.15.178:0/3687565355 >> v1:172.21.15.178:6789/0 conn(0x7f57901076d0 legacy=0x7f5790107ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:13.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.949+0000 7f5796ed6700 1 -- 172.21.15.178:0/3687565355 shutdown_connections 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.949+0000 7f5796ed6700 1 -- 172.21.15.178:0/3687565355 >> 172.21.15.178:0/3687565355 conn(0x7f57900fb700 msgr2=0x7f57900fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.949+0000 7f5796ed6700 1 -- 172.21.15.178:0/3687565355 shutdown_connections 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.949+0000 7f5796ed6700 1 -- 172.21.15.178:0/3687565355 wait complete. 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.949+0000 7f5796ed6700 1 Processor -- start 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.949+0000 7f5796ed6700 1 -- start start 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f5796ed6700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f57901a62e0 con 0x7f57901076d0 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f5796ed6700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f57901a74c0 con 0x7f57901a2610 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f5796ed6700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f57901a86c0 con 0x7f5790196ae0 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f5794c72700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f57901076d0 0x7f57901963d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:54242/0 (socket says 172.21.15.178:54242) 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f5794c72700 1 -- 172.21.15.178:0/290285042 learned_addr learned my addr 172.21.15.178:0/290285042 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406827507 0 0) 0x7f57901a62e0 con 0x7f57901076d0 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f577c003bb0 con 0x7f57901076d0 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445636834 0 0) 0x7f57901a86c0 con 0x7f5790196ae0 2024-02-14T17:38:13.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57901a62e0 con 0x7f5790196ae0 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4192257153 0 0) 0x7f577c003bb0 con 0x7f57901076d0 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f57901a86c0 con 0x7f57901076d0 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f5788001360 con 0x7f57901076d0 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2948411992 0 0) 0x7f57901a62e0 con 0x7f5790196ae0 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f577c003bb0 con 0x7f5790196ae0 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f5778001830 con 0x7f5790196ae0 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577323626 0 0) 0x7f57901a74c0 con 0x7f57901a2610 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57901a62e0 con 0x7f57901a2610 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1105277411 0 0) 0x7f57901a86c0 con 0x7f57901076d0 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 >> v1:172.21.15.178:6790/0 conn(0x7f5790196ae0 legacy=0x7f57901a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 >> v1:172.21.15.184:6789/0 conn(0x7f57901a2610 legacy=0x7f57901a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57901a98c0 con 0x7f57901076d0 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.950+0000 7f5796ed6700 1 -- 172.21.15.178:0/290285042 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57901a88b0 con 0x7f57901076d0 2024-02-14T17:38:13.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.951+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f5788003eb0 con 0x7f57901076d0 2024-02-14T17:38:13.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.951+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f57880052d0 con 0x7f57901076d0 2024-02-14T17:38:13.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.951+0000 7f5796ed6700 1 -- 172.21.15.178:0/290285042 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f57901a8db0 con 0x7f57901076d0 2024-02-14T17:38:13.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.952+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7f5788005510 con 0x7f57901076d0 2024-02-14T17:38:13.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.952+0000 7f5796ed6700 1 -- 172.21.15.178:0/290285042 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57900632a0 con 0x7f57901076d0 2024-02-14T17:38:13.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.952+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2087062522 0 0) 0x7f57880477d0 con 0x7f57901076d0 2024-02-14T17:38:13.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:13.957+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f57880137f0 con 0x7f57901076d0 2024-02-14T17:38:14.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:14.152+0000 7f5796ed6700 1 -- 172.21.15.178:0/290285042 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f57901976e0 con 0x7f57901076d0 2024-02-14T17:38:14.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:14.152+0000 7f578dffb700 1 -- 172.21.15.178:0/290285042 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+193 (unknown 2189217731 0 1027364289) 0x7f5788016b00 con 0x7f57901076d0 2024-02-14T17:38:14.154 INFO:teuthology.orchestra.run.smithi178.stdout:# minimal ceph.conf for 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:38:14.154 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-02-14T17:38:14.154 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:38:14.155 INFO:teuthology.orchestra.run.smithi178.stdout: mon_host = 172.21.15.178:6789/0 172.21.15.184:6789/0 v1:172.21.15.178:6790/0 2024-02-14T17:38:14.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:14.154+0000 7f5796ed6700 1 -- 172.21.15.178:0/290285042 >> v1:172.21.15.178:6800/1500761607 conn(0x7f577c03a580 legacy=0x7f577c03ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:14.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:14.154+0000 7f5796ed6700 1 -- 172.21.15.178:0/290285042 >> v1:172.21.15.178:6789/0 conn(0x7f57901076d0 legacy=0x7f57901963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:14.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:14.154+0000 7f5796ed6700 1 -- 172.21.15.178:0/290285042 shutdown_connections 2024-02-14T17:38:14.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:14.154+0000 7f5796ed6700 1 -- 172.21.15.178:0/290285042 >> 172.21.15.178:0/290285042 conn(0x7f57900fb700 msgr2=0x7f57901089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:38:14.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:14.154+0000 7f5796ed6700 1 -- 172.21.15.178:0/290285042 shutdown_connections 2024-02-14T17:38:14.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:14.154+0000 7f5796ed6700 1 -- 172.21.15.178:0/290285042 wait complete. 2024-02-14T17:38:14.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:14 smithi178 ceph-mon[30321]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T17:38:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:14 smithi178 ceph-mon[30321]: Reconfiguring daemon mon.a on smithi178 2024-02-14T17:38:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/290285042' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:14 smithi178 ceph-mon[39811]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T17:38:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:14 smithi178 ceph-mon[39811]: Reconfiguring daemon mon.a on smithi178 2024-02-14T17:38:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/290285042' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:14 smithi184 ceph-mon[33536]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T17:38:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:14 smithi184 ceph-mon[33536]: Reconfiguring daemon mon.a on smithi178 2024-02-14T17:38:14.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/290285042' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:15.233 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-14T17:38:15.233 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:38:15.233 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2024-02-14T17:38:15.264 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:38:15.264 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:38:15.333 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:38:15.333 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.conf 2024-02-14T17:38:15.368 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:38:15.368 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:38:15.438 INFO:tasks.cephadm:Adding mgr.y on smithi178 2024-02-14T17:38:15.438 INFO:tasks.cephadm:Adding mgr.x on smithi184 2024-02-14T17:38:15.439 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply mgr '2;smithi178=y;smithi184=x' 2024-02-14T17:38:15.708 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:38:15.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:15 smithi184 ceph-mon[33536]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:15.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:15 smithi178 ceph-mon[30321]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:15.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:15 smithi178 ceph-mon[39811]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:16.928 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:16.928 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:16.928 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[30321]: Reconfiguring mon.c (monmap changed)... 2024-02-14T17:38:16.929 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:16.929 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:38:16.929 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:16.929 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[30321]: Reconfiguring daemon mon.c on smithi178 2024-02-14T17:38:16.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:16.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:16.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[39811]: Reconfiguring mon.c (monmap changed)... 2024-02-14T17:38:16.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:16.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:38:16.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:16.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:16 smithi178 ceph-mon[39811]: Reconfiguring daemon mon.c on smithi178 2024-02-14T17:38:16.931 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:16.931 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:16.931 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:16 smithi184 ceph-mon[33536]: Reconfiguring mon.c (monmap changed)... 2024-02-14T17:38:16.931 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:16.931 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:38:16.931 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:16.931 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:16 smithi184 ceph-mon[33536]: Reconfiguring daemon mon.c on smithi178 2024-02-14T17:38:17.269 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.265+0000 7fd75971a700 1 -- 172.21.15.184:0/2659425120 >> v1:172.21.15.178:6789/0 conn(0x7fd754100630 legacy=0x7fd754100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:17.269 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.265+0000 7fd75971a700 1 -- 172.21.15.184:0/2659425120 shutdown_connections 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.265+0000 7fd75971a700 1 -- 172.21.15.184:0/2659425120 >> 172.21.15.184:0/2659425120 conn(0x7fd7540fbbd0 msgr2=0x7fd7540fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.265+0000 7fd75971a700 1 -- 172.21.15.184:0/2659425120 shutdown_connections 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.265+0000 7fd75971a700 1 -- 172.21.15.184:0/2659425120 wait complete. 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.265+0000 7fd75971a700 1 Processor -- start 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.265+0000 7fd75971a700 1 -- start start 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd75971a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd7541a1870 con 0x7fd754196710 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd75971a700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd7541a2a50 con 0x7fd75419dc00 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd75971a700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd7541a3c50 con 0x7fd754100630 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd7537fe700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fd75419dc00 0x7fd75419fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:33908/0 (socket says 172.21.15.184:33908) 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd7537fe700 1 -- 172.21.15.184:0/204250783 learned_addr learned my addr 172.21.15.184:0/204250783 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd7527fc700 1 --1- 172.21.15.184:0/204250783 >> v1:172.21.15.178:6789/0 conn(0x7fd754196710 0x7fd75419c4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.184:58572/0 (socket says 172.21.15.184:58572) 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777142377 0 0) 0x7fd7541a2a50 con 0x7fd75419dc00 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd73c003bb0 con 0x7fd75419dc00 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473558911 0 0) 0x7fd7541a1870 con 0x7fd754196710 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7541a2a50 con 0x7fd754196710 2024-02-14T17:38:17.270 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818329750 0 0) 0x7fd7541a3c50 con 0x7fd754100630 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7541a1870 con 0x7fd754100630 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2965550500 0 0) 0x7fd73c003bb0 con 0x7fd75419dc00 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd7541a3c50 con 0x7fd75419dc00 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd74c001b70 con 0x7fd75419dc00 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3622290624 0 0) 0x7fd7541a2a50 con 0x7fd754196710 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd73c003bb0 con 0x7fd754196710 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd748001810 con 0x7fd754196710 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2413122327 0 0) 0x7fd7541a1870 con 0x7fd754100630 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd7541a2a50 con 0x7fd754100630 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.266+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd744000fe0 con 0x7fd754100630 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.267+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2543681121 0 0) 0x7fd7541a3c50 con 0x7fd75419dc00 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.267+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 >> v1:172.21.15.178:6790/0 conn(0x7fd754100630 legacy=0x7fd754196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.267+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 >> v1:172.21.15.178:6789/0 conn(0x7fd754196710 legacy=0x7fd75419c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.267+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7541a4e50 con 0x7fd75419dc00 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.267+0000 7fd75971a700 1 -- 172.21.15.184:0/204250783 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7541a1a60 con 0x7fd75419dc00 2024-02-14T17:38:17.271 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.267+0000 7fd75971a700 1 -- 172.21.15.184:0/204250783 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7541a1f60 con 0x7fd75419dc00 2024-02-14T17:38:17.272 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.267+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fd74c0051c0 con 0x7fd75419dc00 2024-02-14T17:38:17.272 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.267+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd74c005ff0 con 0x7fd75419dc00 2024-02-14T17:38:17.272 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.267+0000 7fd75971a700 1 -- 172.21.15.184:0/204250783 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd740005150 con 0x7fd75419dc00 2024-02-14T17:38:17.272 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.268+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7fd74c0114e0 con 0x7fd75419dc00 2024-02-14T17:38:17.272 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.268+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2087062522 0 0) 0x7fd74c0472c0 con 0x7fd75419dc00 2024-02-14T17:38:17.274 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.273+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fd74c012ea0 con 0x7fd75419dc00 2024-02-14T17:38:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.488+0000 7fd75971a700 1 -- 172.21.15.184:0/204250783 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi178=y;smithi184=x", "target": ["mon-mgr", ""]}) v1 -- 0x7fd740000bc0 con 0x7fd73c03a510 2024-02-14T17:38:17.495 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.494+0000 7fd73bfff700 1 -- 172.21.15.184:0/204250783 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7fd740000bc0 con 0x7fd73c03a510 2024-02-14T17:38:17.496 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mgr update... 2024-02-14T17:38:17.498 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.496+0000 7fd75971a700 1 -- 172.21.15.184:0/204250783 >> v1:172.21.15.178:6800/1500761607 conn(0x7fd73c03a510 legacy=0x7fd73c03c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:17.498 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.496+0000 7fd75971a700 1 -- 172.21.15.184:0/204250783 >> v1:172.21.15.184:6789/0 conn(0x7fd75419dc00 legacy=0x7fd75419fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:17.498 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.496+0000 7fd75971a700 1 -- 172.21.15.184:0/204250783 shutdown_connections 2024-02-14T17:38:17.498 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.496+0000 7fd75971a700 1 -- 172.21.15.184:0/204250783 >> 172.21.15.184:0/204250783 conn(0x7fd7540fbbd0 msgr2=0x7fd754102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:38:17.498 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.497+0000 7fd75971a700 1 -- 172.21.15.184:0/204250783 shutdown_connections 2024-02-14T17:38:17.498 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:38:17.497+0000 7fd75971a700 1 -- 172.21.15.184:0/204250783 wait complete. 2024-02-14T17:38:17.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:17 smithi184 ceph-mon[33536]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:17 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:17 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:17 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:17 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:38:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:17 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:17 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[30321]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:38:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[39811]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:18.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:38:18.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:38:18.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:18.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:17 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:18.096 DEBUG:teuthology.orchestra.run.smithi184:mgr.x> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mgr.x.service 2024-02-14T17:38:18.098 INFO:tasks.cephadm:Deploying OSDs... 2024-02-14T17:38:18.099 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:38:18.099 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:38:18.117 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-14T17:38:18.117 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-02-14T17:38:18.175 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-14T17:38:18.175 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:38:18.175 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-02-14T17:38:18.175 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:38:18.175 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:38:18.175 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-02-14 17:38:05.080586461 +0000 2024-02-14T17:38:18.175 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-02-14 17:32:08.818492568 +0000 2024-02-14T17:38:18.175 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-02-14 17:32:08.818492568 +0000 2024-02-14T17:38:18.175 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-02-14 17:32:08.818492568 +0000 2024-02-14T17:38:18.176 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-14T17:38:18.245 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-02-14T17:38:18.246 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-02-14T17:38:18.246 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000150672 s, 3.4 MB/s 2024-02-14T17:38:18.247 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-14T17:38:18.305 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-02-14T17:38:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-14T17:38:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:38:18.362 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-14T17:38:18.362 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:38:18.362 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:38:18.362 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-02-14 17:38:05.080586461 +0000 2024-02-14T17:38:18.362 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-02-14 17:32:09.145486486 +0000 2024-02-14T17:38:18.362 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-02-14 17:32:09.145486486 +0000 2024-02-14T17:38:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-02-14 17:32:09.145486486 +0000 2024-02-14T17:38:18.362 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-14T17:38:18.431 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-02-14T17:38:18.431 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-02-14T17:38:18.431 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000128291 s, 4.0 MB/s 2024-02-14T17:38:18.432 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-14T17:38:18.491 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-02-14T17:38:18.548 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-14T17:38:18.548 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:38:18.548 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-14T17:38:18.548 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:38:18.548 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:38:18.548 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-02-14 17:38:05.081586443 +0000 2024-02-14T17:38:18.548 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-02-14 17:32:09.480480255 +0000 2024-02-14T17:38:18.548 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-02-14 17:32:09.480480255 +0000 2024-02-14T17:38:18.548 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-02-14 17:32:09.480480255 +0000 2024-02-14T17:38:18.549 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-14T17:38:18.616 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-02-14T17:38:18.616 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-02-14T17:38:18.616 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 9.8012e-05 s, 5.2 MB/s 2024-02-14T17:38:18.617 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-14T17:38:18.674 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-02-14T17:38:18.730 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-14T17:38:18.730 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:38:18.730 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-14T17:38:18.730 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:38:18.730 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:38:18.730 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-02-14 17:38:05.081586443 +0000 2024-02-14T17:38:18.731 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-02-14 17:32:09.826473819 +0000 2024-02-14T17:38:18.731 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-02-14 17:32:09.826473819 +0000 2024-02-14T17:38:18.731 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-02-14 17:32:09.826473819 +0000 2024-02-14T17:38:18.731 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-14T17:38:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-02-14T17:38:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-02-14T17:38:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000100543 s, 5.1 MB/s 2024-02-14T17:38:18.798 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-14T17:38:18.855 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:38:18.855 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:38:18.870 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-14T17:38:18.870 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-02-14T17:38:18.926 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-14T17:38:18.927 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:38:18.927 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-14T17:38:18.927 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:38:18.927 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:38:18.927 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-02-14 17:37:56.693438836 +0000 2024-02-14T17:38:18.927 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-02-14 17:32:08.800940176 +0000 2024-02-14T17:38:18.927 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-02-14 17:32:08.800940176 +0000 2024-02-14T17:38:18.927 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-02-14 17:32:08.800940176 +0000 2024-02-14T17:38:18.927 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-14T17:38:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:18 smithi184 ceph-mon[33536]: Reconfiguring mon.b (monmap changed)... 2024-02-14T17:38:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:18 smithi184 ceph-mon[33536]: Reconfiguring daemon mon.b on smithi184 2024-02-14T17:38:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:18 smithi184 ceph-mon[33536]: from='client.24106 v1:172.21.15.184:0/204250783' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi178=y;smithi184=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:38:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:18 smithi184 ceph-mon[33536]: Saving service mgr spec with placement smithi178=y;smithi184=x;count:2 2024-02-14T17:38:18.993 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-02-14T17:38:18.993 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-02-14T17:38:18.994 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000205394 s, 2.5 MB/s 2024-02-14T17:38:18.995 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-14T17:38:19.051 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-02-14T17:38:19.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:18 smithi178 ceph-mon[30321]: Reconfiguring mon.b (monmap changed)... 2024-02-14T17:38:19.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:18 smithi178 ceph-mon[30321]: Reconfiguring daemon mon.b on smithi184 2024-02-14T17:38:19.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:18 smithi178 ceph-mon[30321]: from='client.24106 v1:172.21.15.184:0/204250783' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi178=y;smithi184=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:38:19.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:18 smithi178 ceph-mon[30321]: Saving service mgr spec with placement smithi178=y;smithi184=x;count:2 2024-02-14T17:38:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:18 smithi178 ceph-mon[39811]: Reconfiguring mon.b (monmap changed)... 2024-02-14T17:38:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:18 smithi178 ceph-mon[39811]: Reconfiguring daemon mon.b on smithi184 2024-02-14T17:38:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:18 smithi178 ceph-mon[39811]: from='client.24106 v1:172.21.15.184:0/204250783' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi178=y;smithi184=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:38:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:18 smithi178 ceph-mon[39811]: Saving service mgr spec with placement smithi178=y;smithi184=x;count:2 2024-02-14T17:38:19.106 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-14T17:38:19.106 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:38:19.106 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-14T17:38:19.106 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:38:19.106 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:38:19.106 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-02-14 17:37:56.694438815 +0000 2024-02-14T17:38:19.106 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-02-14 17:32:09.154932984 +0000 2024-02-14T17:38:19.107 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-02-14 17:32:09.154932984 +0000 2024-02-14T17:38:19.107 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-02-14 17:32:09.154932984 +0000 2024-02-14T17:38:19.107 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-14T17:38:19.174 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-02-14T17:38:19.174 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-02-14T17:38:19.174 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000181449 s, 2.8 MB/s 2024-02-14T17:38:19.175 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-14T17:38:19.231 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-02-14T17:38:19.285 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-14T17:38:19.286 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:38:19.286 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-14T17:38:19.286 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:38:19.286 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:38:19.286 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-02-14 17:37:56.694438815 +0000 2024-02-14T17:38:19.286 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-02-14 17:32:09.493926096 +0000 2024-02-14T17:38:19.286 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-02-14 17:32:09.493926096 +0000 2024-02-14T17:38:19.286 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-02-14 17:32:09.493926096 +0000 2024-02-14T17:38:19.286 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-14T17:38:19.352 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-02-14T17:38:19.352 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-02-14T17:38:19.352 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00025564 s, 2.0 MB/s 2024-02-14T17:38:19.353 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-14T17:38:19.409 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-02-14T17:38:19.464 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-14T17:38:19.464 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:38:19.464 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-14T17:38:19.464 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:38:19.464 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:38:19.464 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-02-14 17:37:56.694438815 +0000 2024-02-14T17:38:19.464 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-02-14 17:32:09.873918375 +0000 2024-02-14T17:38:19.464 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-02-14 17:32:09.873918375 +0000 2024-02-14T17:38:19.464 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-02-14 17:32:09.873918375 +0000 2024-02-14T17:38:19.465 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-14T17:38:19.530 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-02-14T17:38:19.530 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-02-14T17:38:19.530 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000177709 s, 2.9 MB/s 2024-02-14T17:38:19.531 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-14T17:38:19.586 INFO:tasks.cephadm:Deploying osd.0 on smithi178 with /dev/vg_nvme/lv_4... 2024-02-14T17:38:19.587 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-14T17:38:19.834 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:38:19.926 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:19 smithi184 ceph-mon[33536]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:19.927 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:19 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:19.927 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:19 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:19.927 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:19 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:19.927 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:19 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:19.927 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:19 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:19.927 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:19 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:19.927 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:19 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:38:19.927 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:19 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:38:19.927 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:19 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:38:19.927 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:19 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:19.955 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[30321]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:19.955 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:19.955 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:19.955 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:19.955 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:19.956 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:19.956 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:19.956 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:38:19.956 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:38:19.956 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:38:19.956 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:19.956 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[39811]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:19.956 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:19.956 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:19.956 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:19.956 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:19.957 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:19.957 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:19.957 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:38:19.957 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:38:19.957 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:38:19.957 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:19 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:20.840 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:20 smithi184 ceph-mon[33536]: Deploying daemon mgr.x on smithi184 2024-02-14T17:38:20.893 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:20 smithi178 ceph-mon[39811]: Deploying daemon mgr.x on smithi184 2024-02-14T17:38:20.894 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:20 smithi178 ceph-mon[30321]: Deploying daemon mgr.x on smithi184 2024-02-14T17:38:21.470 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:21 smithi184 podman[35585]: 2024-02-14 17:38:21.400694167 +0000 UTC m=+0.116060982 container create 8a687eaae7d637e516d9bdab7fc7089d859adb935ab00ea5480a9bc3ba512998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-14T17:38:21.772 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-14T17:38:21.772 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-14T17:38:21.773 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-02-14T17:38:21.773 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-02-14T17:38:21.773 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00815124 s, 1.3 GB/s 2024-02-14T17:38:21.773 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 2024-02-14T17:38:21.773 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-02-14T17:38:21.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:21 smithi184 ceph-mon[33536]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:21.970 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:21 smithi184 podman[35585]: 2024-02-14 17:38:21.611930963 +0000 UTC m=+0.327297787 container init 8a687eaae7d637e516d9bdab7fc7089d859adb935ab00ea5480a9bc3ba512998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-14T17:38:21.970 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:21 smithi184 podman[35585]: 2024-02-14 17:38:21.61751301 +0000 UTC m=+0.332879825 container start 8a687eaae7d637e516d9bdab7fc7089d859adb935ab00ea5480a9bc3ba512998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True) 2024-02-14T17:38:21.970 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:21 smithi184 bash[35585]: 8a687eaae7d637e516d9bdab7fc7089d859adb935ab00ea5480a9bc3ba512998 2024-02-14T17:38:21.970 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:21 smithi184 systemd[1]: Started Ceph mgr.x for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:38:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:21 smithi178 ceph-mon[30321]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:21 smithi178 ceph-mon[39811]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:22.417 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2024-02-14T17:38:22.469 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:22 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:22.182+0000 7f531032c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:38:22.470 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:22 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:22.279+0000 7f531032c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:38:22.651 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:38:23.023 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.023 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.023 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.023 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.023 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:23.023 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.023 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.023 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.024 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.024 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:23.075 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.075 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.076 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.076 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.076 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:23.076 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:22 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:22.979+0000 7f531032c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:38:23.076 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:23.073+0000 7f531032c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:38:23.326 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:23.218+0000 7f531032c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:38:23.326 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:23.323+0000 7f531032c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:38:23.676 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:23.407+0000 7f531032c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:38:23.676 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:23.673+0000 7f531032c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:38:23.966 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:23.783+0000 7f531032c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:38:23.966 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:23.869+0000 7f531032c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:38:23.966 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:23.963+0000 7f531032c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:38:23.966 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-mon[33536]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:23.967 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.967 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.967 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:23.967 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:23.967 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:23.967 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:38:23.967 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:38:23.967 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:24.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[30321]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:24.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:24.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:24.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:24.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:24.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:38:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:38:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[39811]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:38:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:38:24.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:24.219 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:24 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:24.140+0000 7f531032c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:38:24.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.919+0000 7fe537938700 1 -- 172.21.15.178:0/2473120579 >> v1:172.21.15.178:6789/0 conn(0x7fe530109c00 legacy=0x7fe53010c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:24.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.919+0000 7fe537938700 1 -- 172.21.15.178:0/2473120579 shutdown_connections 2024-02-14T17:38:24.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.919+0000 7fe537938700 1 -- 172.21.15.178:0/2473120579 >> 172.21.15.178:0/2473120579 conn(0x7fe5300fd070 msgr2=0x7fe5300ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:38:24.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.919+0000 7fe537938700 1 -- 172.21.15.178:0/2473120579 shutdown_connections 2024-02-14T17:38:24.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.919+0000 7fe537938700 1 -- 172.21.15.178:0/2473120579 wait complete. 2024-02-14T17:38:24.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.919+0000 7fe537938700 1 Processor -- start 2024-02-14T17:38:24.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.919+0000 7fe537938700 1 -- start start 2024-02-14T17:38:24.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe537938700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe5301977c0 con 0x7fe530105ec0 2024-02-14T17:38:24.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe537938700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe5301a2fb0 con 0x7fe530101a90 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe537938700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe5301a4150 con 0x7fe530109c00 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe534ed3700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe530105ec0 0x7fe53019d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:59238/0 (socket says 172.21.15.178:59238) 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe534ed3700 1 -- 172.21.15.178:0/3555473060 learned_addr learned my addr 172.21.15.178:0/3555473060 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488226090 0 0) 0x7fe5301977c0 con 0x7fe530105ec0 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5180035b0 con 0x7fe530105ec0 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669524978 0 0) 0x7fe5301a4150 con 0x7fe530109c00 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5301977c0 con 0x7fe530109c00 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210438059 0 0) 0x7fe5301a2fb0 con 0x7fe530101a90 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5301a4150 con 0x7fe530101a90 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2480242535 0 0) 0x7fe5180035b0 con 0x7fe530105ec0 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe5301a2fb0 con 0x7fe530105ec0 2024-02-14T17:38:24.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fe524001930 con 0x7fe530105ec0 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1968796395 0 0) 0x7fe5301977c0 con 0x7fe530109c00 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe5180035b0 con 0x7fe530109c00 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fe5200043e0 con 0x7fe530109c00 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2436908130 0 0) 0x7fe5301a4150 con 0x7fe530101a90 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe5301977c0 con 0x7fe530101a90 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fe528001bb0 con 0x7fe530101a90 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1740109852 0 0) 0x7fe5301a2fb0 con 0x7fe530105ec0 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 >> v1:172.21.15.178:6790/0 conn(0x7fe530109c00 legacy=0x7fe5301a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.920+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 >> v1:172.21.15.184:6789/0 conn(0x7fe530101a90 legacy=0x7fe530196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.921+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5301a52f0 con 0x7fe530105ec0 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.921+0000 7fe537938700 1 -- 172.21.15.178:0/3555473060 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5301a31a0 con 0x7fe530105ec0 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.921+0000 7fe537938700 1 -- 172.21.15.178:0/3555473060 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5301a36a0 con 0x7fe530105ec0 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.921+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fe524001c40 con 0x7fe530105ec0 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.921+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fe524005c40 con 0x7fe530105ec0 2024-02-14T17:38:24.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.921+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 13) v1 ==== 45954+0+0 (unknown 1095177771 0 0) 0x7fe524011210 con 0x7fe530105ec0 2024-02-14T17:38:24.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.922+0000 7fe537938700 1 -- 172.21.15.178:0/3555473060 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c005150 con 0x7fe530105ec0 2024-02-14T17:38:24.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.922+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2087062522 0 0) 0x7fe524047ee0 con 0x7fe530105ec0 2024-02-14T17:38:24.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:24.927+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fe524012d50 con 0x7fe530105ec0 2024-02-14T17:38:25.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:24 smithi178 ceph-mon[30321]: Reconfiguring mgr.y (unknown last config time)... 2024-02-14T17:38:25.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:24 smithi178 ceph-mon[30321]: Reconfiguring daemon mgr.y on smithi178 2024-02-14T17:38:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:24 smithi178 ceph-mon[39811]: Reconfiguring mgr.y (unknown last config time)... 2024-02-14T17:38:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:24 smithi178 ceph-mon[39811]: Reconfiguring daemon mgr.y on smithi178 2024-02-14T17:38:25.105 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:24 smithi184 ceph-mon[33536]: Reconfiguring mgr.y (unknown last config time)... 2024-02-14T17:38:25.105 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:24 smithi184 ceph-mon[33536]: Reconfiguring daemon mgr.y on smithi178 2024-02-14T17:38:25.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:25.135+0000 7fe537938700 1 -- 172.21.15.178:0/3555473060 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe51c000bc0 con 0x7fe51803a540 2024-02-14T17:38:25.469 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:25.102+0000 7f531032c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:38:25.470 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:25.231+0000 7f531032c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:38:25.820 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:25.562+0000 7f531032c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:38:25.820 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:25.650+0000 7f531032c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:38:26.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[30321]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:26.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:38:26.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:38:26.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:26.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:26.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:26.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:26.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:26.019 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[39811]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:26.019 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:38:26.019 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:38:26.020 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:26.020 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:26.020 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:26.020 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:26.020 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:25 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-mon[33536]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:38:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:38:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:26.221 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:25 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:25.845+0000 7f531032c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:38:26.221 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:26 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:26.028+0000 7f531032c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:38:26.569 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:26 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:26.308+0000 7f531032c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:38:26.570 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:26 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:26.567+0000 7f531032c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:38:26.821 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:26 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:26.799+0000 7f531032c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:38:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:26 smithi178 ceph-mon[30321]: from='client.14217 v1:172.21.15.178:0/3555473060' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:38:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:26 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:26 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:26 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:26 smithi178 ceph-mon[39811]: from='client.14217 v1:172.21.15.178:0/3555473060' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:38:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:26 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:26 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:26 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:27.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:26 smithi184 ceph-mon[33536]: from='client.14217 v1:172.21.15.178:0/3555473060' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:38:27.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:26 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:27.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:26 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:27.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:26 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:27 smithi184 ceph-mon[33536]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/13073357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96b590dc-522b-4b8d-938b-5d3de2944e53"}]: dispatch 2024-02-14T17:38:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:27 smithi184 ceph-mon[33536]: from='client.34118 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96b590dc-522b-4b8d-938b-5d3de2944e53"}]: dispatch 2024-02-14T17:38:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:27 smithi184 ceph-mon[33536]: from='client.34118 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96b590dc-522b-4b8d-938b-5d3de2944e53"}]': finished 2024-02-14T17:38:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:27 smithi184 ceph-mon[33536]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T17:38:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:27 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:27.970 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:27 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:27.588+0000 7f531032c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:38:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[39811]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/13073357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96b590dc-522b-4b8d-938b-5d3de2944e53"}]: dispatch 2024-02-14T17:38:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[39811]: from='client.34118 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96b590dc-522b-4b8d-938b-5d3de2944e53"}]: dispatch 2024-02-14T17:38:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[39811]: from='client.34118 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96b590dc-522b-4b8d-938b-5d3de2944e53"}]': finished 2024-02-14T17:38:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[39811]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T17:38:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[30321]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/13073357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96b590dc-522b-4b8d-938b-5d3de2944e53"}]: dispatch 2024-02-14T17:38:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[30321]: from='client.34118 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96b590dc-522b-4b8d-938b-5d3de2944e53"}]: dispatch 2024-02-14T17:38:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[30321]: from='client.34118 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96b590dc-522b-4b8d-938b-5d3de2944e53"}]': finished 2024-02-14T17:38:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[30321]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T17:38:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:27 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:28.455 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:28 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:28.149+0000 7f531032c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:38:28.455 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:28 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:28.243+0000 7f531032c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:38:28.455 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:38:28 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:38:28.452+0000 7f531032c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:38:28.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:28.828+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mgrmap(e 14) v1 ==== 91316+0+0 (unknown 3490801675 0 0) 0x7fe5240468c0 con 0x7fe530105ec0 2024-02-14T17:38:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1041897625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:38:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[30321]: Standby manager daemon x started 2024-02-14T17:38:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[30321]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:38:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[30321]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:38:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[30321]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:38:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[30321]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:38:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1041897625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:38:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[39811]: Standby manager daemon x started 2024-02-14T17:38:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[39811]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:38:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[39811]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:38:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[39811]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:38:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:28 smithi178 ceph-mon[39811]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:38:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1041897625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:38:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:28 smithi184 ceph-mon[33536]: Standby manager daemon x started 2024-02-14T17:38:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:28 smithi184 ceph-mon[33536]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:38:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:28 smithi184 ceph-mon[33536]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:38:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:28 smithi184 ceph-mon[33536]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:38:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:28 smithi184 ceph-mon[33536]: from='mgr.? v1:172.21.15.184:0/267318974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:38:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:29 smithi178 ceph-mon[39811]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:29 smithi178 ceph-mon[39811]: mgrmap e14: y(active, since 68s), standbys: x 2024-02-14T17:38:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:29 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:38:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:29 smithi178 ceph-mon[30321]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:29 smithi178 ceph-mon[30321]: mgrmap e14: y(active, since 68s), standbys: x 2024-02-14T17:38:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:29 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:38:30.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:29 smithi184 ceph-mon[33536]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:30.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:29 smithi184 ceph-mon[33536]: mgrmap e14: y(active, since 68s), standbys: x 2024-02-14T17:38:30.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:29 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:38:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:31 smithi178 ceph-mon[30321]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:31 smithi178 ceph-mon[39811]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:32.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:31 smithi184 ceph-mon[33536]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:33 smithi178 ceph-mon[30321]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:33 smithi178 ceph-mon[39811]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:33 smithi184 ceph-mon[33536]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:34.978 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:34 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T17:38:34.979 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:34 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:34.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:34 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T17:38:34.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:34 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:35.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:34 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T17:38:35.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:34 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:36.042 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:35 smithi178 ceph-mon[30321]: Deploying daemon osd.0 on smithi178 2024-02-14T17:38:36.042 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:35 smithi178 ceph-mon[30321]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:36.042 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:35 smithi178 ceph-mon[39811]: Deploying daemon osd.0 on smithi178 2024-02-14T17:38:36.043 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:35 smithi178 ceph-mon[39811]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:36.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:35 smithi184 ceph-mon[33536]: Deploying daemon osd.0 on smithi178 2024-02-14T17:38:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:35 smithi184 ceph-mon[33536]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:37.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:37 smithi178 ceph-mon[30321]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:37.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:37 smithi178 ceph-mon[39811]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:38.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:37 smithi184 ceph-mon[33536]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:39 smithi184 ceph-mon[33536]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:40.252 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:39 smithi178 ceph-mon[39811]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:40.252 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:40.252 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:40.252 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:40.252 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:39 smithi178 ceph-mon[30321]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:40.252 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:40.252 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:40.253 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:42.151 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:41 smithi178 ceph-mon[30321]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:42.152 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:41 smithi178 ceph-mon[39811]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:42.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:41 smithi184 ceph-mon[33536]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:42.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:42 smithi184 ceph-mon[33536]: from='osd.0 v1:172.21.15.178:6801/2284738682' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:38:42.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:42 smithi184 ceph-mon[33536]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:38:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:42 smithi178 ceph-mon[30321]: from='osd.0 v1:172.21.15.178:6801/2284738682' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:38:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:42 smithi178 ceph-mon[30321]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:38:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:42 smithi178 ceph-mon[39811]: from='osd.0 v1:172.21.15.178:6801/2284738682' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:38:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:42 smithi178 ceph-mon[39811]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:38:43.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:43.191+0000 7fe52e7fc700 1 -- 172.21.15.178:0/3555473060 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 97586898) 0x7fe51c000bc0 con 0x7fe51803a540 2024-02-14T17:38:43.194 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 0 on host 'smithi178' 2024-02-14T17:38:43.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:43.195+0000 7fe537938700 1 -- 172.21.15.178:0/3555473060 >> v1:172.21.15.178:6800/1500761607 conn(0x7fe51803a540 legacy=0x7fe51803ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:43.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:43.195+0000 7fe537938700 1 -- 172.21.15.178:0/3555473060 >> v1:172.21.15.178:6789/0 conn(0x7fe530105ec0 legacy=0x7fe53019d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:43.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:43.195+0000 7fe537938700 1 -- 172.21.15.178:0/3555473060 shutdown_connections 2024-02-14T17:38:43.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:43.195+0000 7fe537938700 1 -- 172.21.15.178:0/3555473060 >> 172.21.15.178:0/3555473060 conn(0x7fe5300fd070 msgr2=0x7fe530108a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:38:43.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:43.195+0000 7fe537938700 1 -- 172.21.15.178:0/3555473060 shutdown_connections 2024-02-14T17:38:43.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:43.195+0000 7fe537938700 1 -- 172.21.15.178:0/3555473060 wait complete. 2024-02-14T17:38:43.721 DEBUG:teuthology.orchestra.run.smithi178:osd.0> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.0.service 2024-02-14T17:38:43.723 INFO:tasks.cephadm:Deploying osd.1 on smithi178 with /dev/vg_nvme/lv_3... 2024-02-14T17:38:43.723 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-14T17:38:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:38:44.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[30321]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:44.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[30321]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T17:38:44.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[30321]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T17:38:44.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[30321]: from='osd.0 v1:172.21.15.178:6801/2284738682' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:38:44.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:44.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[30321]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:38:44.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:44.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[39811]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:44.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[39811]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T17:38:44.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[39811]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T17:38:44.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[39811]: from='osd.0 v1:172.21.15.178:6801/2284738682' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:38:44.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:44.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[39811]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:38:44.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:44.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:44.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:43 smithi184 ceph-mon[33536]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:43 smithi184 ceph-mon[33536]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T17:38:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:43 smithi184 ceph-mon[33536]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T17:38:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:43 smithi184 ceph-mon[33536]: from='osd.0 v1:172.21.15.178:6801/2284738682' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:38:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:43 smithi184 ceph-mon[33536]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:38:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:44.458 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0[44215]: 2024-02-14T17:38:44.113+0000 7efd4a47f700 -1 osd.0 0 waiting for initial osdmap 2024-02-14T17:38:44.458 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0[44215]: 2024-02-14T17:38:44.115+0000 7efd4226a700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:38:45.111 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[30321]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:38:45.111 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[30321]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T17:38:45.111 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:45.111 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:45.111 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:45.111 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:45.111 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[30321]: from='osd.0 ' entity='osd.0' 2024-02-14T17:38:45.111 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[39811]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:38:45.111 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[39811]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T17:38:45.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:45.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:45.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:45.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:45.112 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:44 smithi178 ceph-mon[39811]: from='osd.0 ' entity='osd.0' 2024-02-14T17:38:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:44 smithi184 ceph-mon[33536]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:38:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:44 smithi184 ceph-mon[33536]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T17:38:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:44 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:44 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:44 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:44 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:44 smithi184 ceph-mon[33536]: from='osd.0 ' entity='osd.0' 2024-02-14T17:38:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[30321]: purged_snaps scrub starts 2024-02-14T17:38:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[30321]: purged_snaps scrub ok 2024-02-14T17:38:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[30321]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[30321]: osd.0 v1:172.21.15.178:6801/2284738682 boot 2024-02-14T17:38:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[30321]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T17:38:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[39811]: purged_snaps scrub starts 2024-02-14T17:38:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[39811]: purged_snaps scrub ok 2024-02-14T17:38:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[39811]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[39811]: osd.0 v1:172.21.15.178:6801/2284738682 boot 2024-02-14T17:38:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[39811]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T17:38:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:46.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:45 smithi184 ceph-mon[33536]: purged_snaps scrub starts 2024-02-14T17:38:46.251 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:45 smithi184 ceph-mon[33536]: purged_snaps scrub ok 2024-02-14T17:38:46.252 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:45 smithi184 ceph-mon[33536]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:38:46.252 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:46.252 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:45 smithi184 ceph-mon[33536]: osd.0 v1:172.21.15.178:6801/2284738682 boot 2024-02-14T17:38:46.252 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:45 smithi184 ceph-mon[33536]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T17:38:46.252 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:38:46.252 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-14T17:38:46.252 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-14T17:38:46.253 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-02-14T17:38:46.253 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-02-14T17:38:46.253 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0105585 s, 993 MB/s 2024-02-14T17:38:46.253 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-02-14T17:38:46.875 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2024-02-14T17:38:47.343 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:38:47.433 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:47 smithi178 ceph-mon[39811]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T17:38:47.433 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:47 smithi178 ceph-mon[39811]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:47.433 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:47 smithi178 ceph-mon[30321]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T17:38:47.433 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:47 smithi178 ceph-mon[30321]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:47.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:47 smithi184 ceph-mon[33536]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T17:38:47.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:47 smithi184 ceph-mon[33536]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[30321]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:38:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:49.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:49.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[39811]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:49.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:49.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:49.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:38:49.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:49.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:49.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:49.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:49.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:49.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.858+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/3655693181 >> v1:172.21.15.178:6790/0 conn(0x7fdcb4101a60 legacy=0x7fdcb4101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:49.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.858+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/3655693181 shutdown_connections 2024-02-14T17:38:49.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.858+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/3655693181 >> 172.21.15.178:0/3655693181 conn(0x7fdcb40fd020 msgr2=0x7fdcb40ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:38:49.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.858+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/3655693181 shutdown_connections 2024-02-14T17:38:49.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.858+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/3655693181 wait complete. 2024-02-14T17:38:49.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.858+0000 7fdcb9bc3700 1 Processor -- start 2024-02-14T17:38:49.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.858+0000 7fdcb9bc3700 1 -- start start 2024-02-14T17:38:49.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb9bc3700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdcb41a3ea0 con 0x7fdcb4105e90 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb9bc3700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdcb41a50a0 con 0x7fdcb4101a60 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb9bc3700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdcb41a62a0 con 0x7fdcb4109bd0 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb3fff700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7fdcb4109bd0 0x7fdcb41a25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:41382/0 (socket says 172.21.15.178:41382) 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcaaffd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fdcb4105e90 0x7fdcb419ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:52006/0 (socket says 172.21.15.178:52006) 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb3fff700 1 -- 172.21.15.178:0/1832382584 learned_addr learned my addr 172.21.15.178:0/1832382584 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285879597 0 0) 0x7fdcb41a3ea0 con 0x7fdcb4105e90 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdca00049e0 con 0x7fdcb4105e90 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910638237 0 0) 0x7fdcb41a62a0 con 0x7fdcb4109bd0 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb41a3ea0 con 0x7fdcb4109bd0 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653038889 0 0) 0x7fdcb41a50a0 con 0x7fdcb4101a60 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb41a62a0 con 0x7fdcb4101a60 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2190795242 0 0) 0x7fdca00049e0 con 0x7fdcb4105e90 2024-02-14T17:38:49.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdcb41a50a0 con 0x7fdcb4105e90 2024-02-14T17:38:49.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fdc9c0018b0 con 0x7fdcb4105e90 2024-02-14T17:38:49.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2736978549 0 0) 0x7fdcb41a3ea0 con 0x7fdcb4109bd0 2024-02-14T17:38:49.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdca00049e0 con 0x7fdcb4109bd0 2024-02-14T17:38:49.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1218084605 0 0) 0x7fdcb41a62a0 con 0x7fdcb4101a60 2024-02-14T17:38:49.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdcb41a3ea0 con 0x7fdcb4101a60 2024-02-14T17:38:49.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fdcac001d70 con 0x7fdcb4109bd0 2024-02-14T17:38:49.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.859+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fdca40044e0 con 0x7fdcb4101a60 2024-02-14T17:38:49.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.860+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2051252699 0 0) 0x7fdcb41a50a0 con 0x7fdcb4105e90 2024-02-14T17:38:49.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.860+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 >> v1:172.21.15.178:6790/0 conn(0x7fdcb4109bd0 legacy=0x7fdcb41a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:49.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.860+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 >> v1:172.21.15.184:6789/0 conn(0x7fdcb4101a60 legacy=0x7fdcb4075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:38:49.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.860+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb41a74a0 con 0x7fdcb4105e90 2024-02-14T17:38:49.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.860+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/1832382584 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb41a4090 con 0x7fdcb4105e90 2024-02-14T17:38:49.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.860+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/1832382584 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb41a4590 con 0x7fdcb4105e90 2024-02-14T17:38:49.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.860+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fdc9c003b30 con 0x7fdcb4105e90 2024-02-14T17:38:49.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.860+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fdc9c005bc0 con 0x7fdcb4105e90 2024-02-14T17:38:49.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.861+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/1832382584 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc94005150 con 0x7fdcb4105e90 2024-02-14T17:38:49.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.861+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 14) v1 ==== 91316+0+0 (unknown 3490801675 0 0) 0x7fdc9c0036a0 con 0x7fdcb4105e90 2024-02-14T17:38:49.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.861+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(9..9 src has 1..9) v4 ==== 1541+0+0 (unknown 175585357 0 0) 0x7fdc9c0874c0 con 0x7fdcb4105e90 2024-02-14T17:38:49.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:49.866+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fdc9c051f50 con 0x7fdcb4105e90 2024-02-14T17:38:50.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:38:50.070+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/1832382584 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc94000bc0 con 0x7fdca006e990 2024-02-14T17:38:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:49 smithi184 ceph-mon[33536]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:50.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:38:50.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:50.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:50.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:50.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:50.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:38:50.934 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[30321]: Detected new or changed devices on smithi178 2024-02-14T17:38:50.934 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi178 to 16064M 2024-02-14T17:38:50.934 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:38:50.934 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[39811]: Detected new or changed devices on smithi178 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi178 to 16064M 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:50.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:50 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:50 smithi184 ceph-mon[33536]: Detected new or changed devices on smithi178 2024-02-14T17:38:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:50 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi178 to 16064M 2024-02-14T17:38:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:50 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:38:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:50 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:38:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:50 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:50 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:38:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:50 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:38:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:50 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:38:51.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:51 smithi178 ceph-mon[39811]: from='client.14244 v1:172.21.15.178:0/1832382584' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:38:51.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:51 smithi178 ceph-mon[39811]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:51.954 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:51 smithi178 ceph-mon[30321]: from='client.14244 v1:172.21.15.178:0/1832382584' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:38:51.954 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:51 smithi178 ceph-mon[30321]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:51 smithi184 ceph-mon[33536]: from='client.14244 v1:172.21.15.178:0/1832382584' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:38:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:51 smithi184 ceph-mon[33536]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1639036502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c97cb851-c6dd-4230-ace7-1fe481f29ae6"}]: dispatch 2024-02-14T17:38:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:52 smithi178 ceph-mon[30321]: from='client.34145 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c97cb851-c6dd-4230-ace7-1fe481f29ae6"}]: dispatch 2024-02-14T17:38:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:52 smithi178 ceph-mon[30321]: from='client.34145 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c97cb851-c6dd-4230-ace7-1fe481f29ae6"}]': finished 2024-02-14T17:38:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:52 smithi178 ceph-mon[30321]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T17:38:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:52 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:38:53.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1639036502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c97cb851-c6dd-4230-ace7-1fe481f29ae6"}]: dispatch 2024-02-14T17:38:53.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:52 smithi178 ceph-mon[39811]: from='client.34145 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c97cb851-c6dd-4230-ace7-1fe481f29ae6"}]: dispatch 2024-02-14T17:38:53.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:52 smithi178 ceph-mon[39811]: from='client.34145 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c97cb851-c6dd-4230-ace7-1fe481f29ae6"}]': finished 2024-02-14T17:38:53.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:52 smithi178 ceph-mon[39811]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T17:38:53.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:52 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:38:53.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1639036502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c97cb851-c6dd-4230-ace7-1fe481f29ae6"}]: dispatch 2024-02-14T17:38:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:52 smithi184 ceph-mon[33536]: from='client.34145 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c97cb851-c6dd-4230-ace7-1fe481f29ae6"}]: dispatch 2024-02-14T17:38:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:52 smithi184 ceph-mon[33536]: from='client.34145 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c97cb851-c6dd-4230-ace7-1fe481f29ae6"}]': finished 2024-02-14T17:38:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:52 smithi184 ceph-mon[33536]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T17:38:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:52 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:38:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:53 smithi178 ceph-mon[30321]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:53 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1248954492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:38:54.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:53 smithi178 ceph-mon[39811]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:54.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:53 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1248954492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:38:54.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:53 smithi184 ceph-mon[33536]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:54.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:53 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1248954492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:38:55.995 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:55 smithi178 ceph-mon[39811]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:55.996 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:55 smithi178 ceph-mon[30321]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:56.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:55 smithi184 ceph-mon[33536]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:57 smithi178 ceph-mon[30321]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:57 smithi178 ceph-mon[39811]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:38:58.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:57 smithi184 ceph-mon[33536]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:00.070 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:59 smithi178 ceph-mon[30321]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:00.070 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:59 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T17:39:00.070 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:38:59 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:00.070 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:59 smithi178 ceph-mon[39811]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:00.070 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:59 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T17:39:00.070 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:38:59 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:00.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:59 smithi184 ceph-mon[33536]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:00.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:59 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T17:39:00.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:38:59 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:01.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:00 smithi178 ceph-mon[30321]: Deploying daemon osd.1 on smithi178 2024-02-14T17:39:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:00 smithi178 ceph-mon[39811]: Deploying daemon osd.1 on smithi178 2024-02-14T17:39:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:00 smithi184 ceph-mon[33536]: Deploying daemon osd.1 on smithi178 2024-02-14T17:39:02.038 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:01 smithi178 ceph-mon[39811]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:02.038 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:01 smithi178 ceph-mon[30321]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:02.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:01 smithi184 ceph-mon[33536]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:03 smithi178 ceph-mon[30321]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:04.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:03 smithi178 ceph-mon[39811]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:04.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:03 smithi184 ceph-mon[33536]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:06.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:05 smithi184 ceph-mon[33536]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:06.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:06.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:06.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:06.296 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:05 smithi178 ceph-mon[30321]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:06.296 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:06.296 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:06.296 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:06.297 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:05 smithi178 ceph-mon[39811]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:06.297 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:06.297 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:06.297 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:08.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:07 smithi184 ceph-mon[33536]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:08.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:07 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:08.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:07 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:08.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:07 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:08.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:07 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:08.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:07 smithi178 ceph-mon[39811]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:08.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:07 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:08.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:07 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:08.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:07 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:08.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:07 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:08.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:07 smithi178 ceph-mon[30321]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:08.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:07 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:08.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:07 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:08.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:07 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:08.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:07 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:08.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:08.625+0000 7fdcb17fa700 1 -- 172.21.15.178:0/1832382584 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1506074726) 0x7fdc94000bc0 con 0x7fdca006e990 2024-02-14T17:39:08.627 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 1 on host 'smithi178' 2024-02-14T17:39:08.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:08.627+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/1832382584 >> v1:172.21.15.178:6800/1500761607 conn(0x7fdca006e990 legacy=0x7fdca0070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:08.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:08.627+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/1832382584 >> v1:172.21.15.178:6789/0 conn(0x7fdcb4105e90 legacy=0x7fdcb419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:08.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:08.627+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/1832382584 shutdown_connections 2024-02-14T17:39:08.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:08.627+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/1832382584 >> 172.21.15.178:0/1832382584 conn(0x7fdcb40fd020 msgr2=0x7fdcb4108a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:08.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:08.627+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/1832382584 shutdown_connections 2024-02-14T17:39:08.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:08.627+0000 7fdcb9bc3700 1 -- 172.21.15.178:0/1832382584 wait complete. 2024-02-14T17:39:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:08 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:08 smithi184 ceph-mon[33536]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:39:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:08 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:08 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:08 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:09.227 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:08 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:09.227 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:08 smithi178 ceph-mon[30321]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:39:09.227 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:08 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:09.227 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:08 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:09.227 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:08 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:09.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:08 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:09.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:08 smithi178 ceph-mon[39811]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:39:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:08 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:08 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:08 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:09.256 DEBUG:teuthology.orchestra.run.smithi178:osd.1> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.1.service 2024-02-14T17:39:09.258 INFO:tasks.cephadm:Deploying osd.2 on smithi178 with /dev/vg_nvme/lv_2... 2024-02-14T17:39:09.258 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-14T17:39:09.997 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:39:10.115 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[30321]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:10.115 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[30321]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T17:39:10.115 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[30321]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T17:39:10.115 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[30321]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:39:10.116 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:10.116 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[30321]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:39:10.116 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[39811]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:10.116 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[39811]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T17:39:10.116 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[39811]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T17:39:10.116 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[39811]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:39:10.116 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:10.116 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:09 smithi178 ceph-mon[39811]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:39:10.117 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1[50608]: 2024-02-14T17:39:10.093+0000 7f33df4ca700 -1 osd.1 0 waiting for initial osdmap 2024-02-14T17:39:10.117 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1[50608]: 2024-02-14T17:39:10.096+0000 7f33d72b5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:39:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:09 smithi184 ceph-mon[33536]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:10.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:09 smithi184 ceph-mon[33536]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T17:39:10.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:09 smithi184 ceph-mon[33536]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T17:39:10.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:09 smithi184 ceph-mon[33536]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:39:10.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:09 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:10.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:09 smithi184 ceph-mon[33536]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:39:11.046 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-mon[30321]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T17:39:11.046 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:11.046 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:11.046 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-mon[30321]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' 2024-02-14T17:39:11.046 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:11.047 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-mon[39811]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T17:39:11.047 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:11.047 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:11.047 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-mon[39811]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' 2024-02-14T17:39:11.047 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:10 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:11.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:10 smithi184 ceph-mon[33536]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T17:39:11.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:11.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:11.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:10 smithi184 ceph-mon[33536]: from='osd.1 v1:172.21.15.178:6805/3678228960' entity='osd.1' 2024-02-14T17:39:11.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:10 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:12.029 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[30321]: purged_snaps scrub starts 2024-02-14T17:39:12.029 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[30321]: purged_snaps scrub ok 2024-02-14T17:39:12.029 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[30321]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:12.029 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[30321]: osd.1 v1:172.21.15.178:6805/3678228960 boot 2024-02-14T17:39:12.029 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[30321]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T17:39:12.030 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:12.030 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[39811]: purged_snaps scrub starts 2024-02-14T17:39:12.030 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[39811]: purged_snaps scrub ok 2024-02-14T17:39:12.030 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[39811]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:12.030 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[39811]: osd.1 v1:172.21.15.178:6805/3678228960 boot 2024-02-14T17:39:12.030 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[39811]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T17:39:12.030 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:11 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:11 smithi184 ceph-mon[33536]: purged_snaps scrub starts 2024-02-14T17:39:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:11 smithi184 ceph-mon[33536]: purged_snaps scrub ok 2024-02-14T17:39:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:11 smithi184 ceph-mon[33536]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:39:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:11 smithi184 ceph-mon[33536]: osd.1 v1:172.21.15.178:6805/3678228960 boot 2024-02-14T17:39:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:11 smithi184 ceph-mon[33536]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T17:39:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:11 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:39:12.592 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-14T17:39:12.592 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-14T17:39:12.592 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-02-14T17:39:12.593 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-02-14T17:39:12.593 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00971938 s, 1.1 GB/s 2024-02-14T17:39:12.593 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-02-14T17:39:13.830 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2024-02-14T17:39:14.070 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:39:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[30321]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[30321]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T17:39:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:14.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[39811]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:14.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[39811]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T17:39:14.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:13 smithi184 ceph-mon[33536]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:13 smithi184 ceph-mon[33536]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T17:39:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:15.202 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:14 smithi178 ceph-mon[39811]: Detected new or changed devices on smithi178 2024-02-14T17:39:15.202 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:14 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi178 to 8032M 2024-02-14T17:39:15.202 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:14 smithi178 ceph-mon[30321]: Detected new or changed devices on smithi178 2024-02-14T17:39:15.202 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:14 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi178 to 8032M 2024-02-14T17:39:15.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:14 smithi184 ceph-mon[33536]: Detected new or changed devices on smithi178 2024-02-14T17:39:15.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:14 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi178 to 8032M 2024-02-14T17:39:15.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.708+0000 7f748fc1f700 1 -- 172.21.15.178:0/1485600319 >> v1:172.21.15.178:6790/0 conn(0x7f7488101f30 legacy=0x7f7488102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:15.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748fc1f700 1 -- 172.21.15.178:0/1485600319 shutdown_connections 2024-02-14T17:39:15.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748fc1f700 1 -- 172.21.15.178:0/1485600319 >> 172.21.15.178:0/1485600319 conn(0x7f74880fb6e0 msgr2=0x7f74880fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:15.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748fc1f700 1 -- 172.21.15.178:0/1485600319 shutdown_connections 2024-02-14T17:39:15.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748fc1f700 1 -- 172.21.15.178:0/1485600319 wait complete. 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748fc1f700 1 Processor -- start 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748fc1f700 1 -- start start 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748fc1f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7488199f80 con 0x7f7488108ae0 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748fc1f700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f74881a5230 con 0x7f7488104eb0 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748fc1f700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f74881a63d0 con 0x7f7488101f30 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748d9bb700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f7488101f30 0x7f7488199430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:50938/0 (socket says 172.21.15.178:50938) 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748e1bc700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f7488108ae0 0x7f74881a2b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:35350/0 (socket says 172.21.15.178:35350) 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.709+0000 7f748d9bb700 1 -- 172.21.15.178:0/338029410 learned_addr learned my addr 172.21.15.178:0/338029410 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456555060 0 0) 0x7f7488199f80 con 0x7f7488108ae0 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74740035b0 con 0x7f7488108ae0 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166822469 0 0) 0x7f74881a63d0 con 0x7f7488101f30 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7488199f80 con 0x7f7488101f30 2024-02-14T17:39:15.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190311143 0 0) 0x7f74881a5230 con 0x7f7488104eb0 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74881a63d0 con 0x7f7488104eb0 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1331514378 0 0) 0x7f74740035b0 con 0x7f7488108ae0 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74881a5230 con 0x7f7488108ae0 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f74800013e0 con 0x7f7488108ae0 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4062816294 0 0) 0x7f7488199f80 con 0x7f7488101f30 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74740035b0 con 0x7f7488101f30 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f7484004430 con 0x7f7488101f30 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3791070478 0 0) 0x7f74881a63d0 con 0x7f7488104eb0 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7488199f80 con 0x7f7488104eb0 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f7478001830 con 0x7f7488104eb0 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1532002197 0 0) 0x7f74881a5230 con 0x7f7488108ae0 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 >> v1:172.21.15.178:6790/0 conn(0x7f7488101f30 legacy=0x7f7488199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 >> v1:172.21.15.184:6789/0 conn(0x7f7488104eb0 legacy=0x7f748819f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:15.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74881a75b0 con 0x7f7488108ae0 2024-02-14T17:39:15.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f748fc1f700 1 -- 172.21.15.178:0/338029410 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74881a4280 con 0x7f7488108ae0 2024-02-14T17:39:15.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f748fc1f700 1 -- 172.21.15.178:0/338029410 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f74881a4780 con 0x7f7488108ae0 2024-02-14T17:39:15.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f7480002060 con 0x7f7488108ae0 2024-02-14T17:39:15.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.710+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f7480005e20 con 0x7f7488108ae0 2024-02-14T17:39:15.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.712+0000 7f748fc1f700 1 -- 172.21.15.178:0/338029410 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7488068560 con 0x7f7488108ae0 2024-02-14T17:39:15.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.712+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 14) v1 ==== 91316+0+0 (unknown 3490801675 0 0) 0x7f7480003a00 con 0x7f7488108ae0 2024-02-14T17:39:15.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.713+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 1833+0+0 (unknown 1270122893 0 0) 0x7f74800878d0 con 0x7f7488108ae0 2024-02-14T17:39:15.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.717+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f74800521e0 con 0x7f7488108ae0 2024-02-14T17:39:15.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:15.923+0000 7f748fc1f700 1 -- 172.21.15.178:0/338029410 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f74880fccb0 con 0x7f747406ea00 2024-02-14T17:39:16.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:15 smithi184 ceph-mon[33536]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:16.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:15 smithi178 ceph-mon[30321]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:16.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:15 smithi178 ceph-mon[39811]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:17.210 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:39:17.210 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:39:17.210 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:17.211 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:39:17.211 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:39:17.211 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:39:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:39:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:17 smithi178 ceph-mon[30321]: from='client.14271 v1:172.21.15.178:0/338029410' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:39:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:17 smithi178 ceph-mon[30321]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:17 smithi178 ceph-mon[39811]: from='client.14271 v1:172.21.15.178:0/338029410' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:39:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:17 smithi178 ceph-mon[39811]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:18.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:17 smithi184 ceph-mon[33536]: from='client.14271 v1:172.21.15.178:0/338029410' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:39:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:17 smithi184 ceph-mon[33536]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:19.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1127679095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bebc9b6f-6349-4f8e-83b1-4177c816c21f"}]: dispatch 2024-02-14T17:39:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:18 smithi184 ceph-mon[33536]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bebc9b6f-6349-4f8e-83b1-4177c816c21f"}]: dispatch 2024-02-14T17:39:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:18 smithi184 ceph-mon[33536]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bebc9b6f-6349-4f8e-83b1-4177c816c21f"}]': finished 2024-02-14T17:39:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:18 smithi184 ceph-mon[33536]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T17:39:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:18 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:19.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1127679095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bebc9b6f-6349-4f8e-83b1-4177c816c21f"}]: dispatch 2024-02-14T17:39:19.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:18 smithi178 ceph-mon[30321]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bebc9b6f-6349-4f8e-83b1-4177c816c21f"}]: dispatch 2024-02-14T17:39:19.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:18 smithi178 ceph-mon[30321]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bebc9b6f-6349-4f8e-83b1-4177c816c21f"}]': finished 2024-02-14T17:39:19.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:18 smithi178 ceph-mon[30321]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T17:39:19.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:18 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:19.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1127679095' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bebc9b6f-6349-4f8e-83b1-4177c816c21f"}]: dispatch 2024-02-14T17:39:19.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:18 smithi178 ceph-mon[39811]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bebc9b6f-6349-4f8e-83b1-4177c816c21f"}]: dispatch 2024-02-14T17:39:19.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:18 smithi178 ceph-mon[39811]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bebc9b6f-6349-4f8e-83b1-4177c816c21f"}]': finished 2024-02-14T17:39:19.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:18 smithi178 ceph-mon[39811]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T17:39:19.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:18 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:19 smithi184 ceph-mon[33536]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3685984616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:39:20.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:19 smithi178 ceph-mon[30321]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:20.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3685984616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:39:20.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:19 smithi178 ceph-mon[39811]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:20.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3685984616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:39:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:21 smithi178 ceph-mon[30321]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:21 smithi178 ceph-mon[39811]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:21 smithi184 ceph-mon[33536]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:24.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:23 smithi178 ceph-mon[30321]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:23 smithi178 ceph-mon[39811]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:24.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:23 smithi184 ceph-mon[33536]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:26.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:25 smithi178 ceph-mon[30321]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:26.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:25 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T17:39:26.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:25 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:25 smithi178 ceph-mon[39811]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:25 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T17:39:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:25 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:25 smithi184 ceph-mon[33536]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:25 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T17:39:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:25 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:27.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:26 smithi184 ceph-mon[33536]: Deploying daemon osd.2 on smithi178 2024-02-14T17:39:27.222 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:26 smithi178 ceph-mon[30321]: Deploying daemon osd.2 on smithi178 2024-02-14T17:39:27.222 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:26 smithi178 ceph-mon[39811]: Deploying daemon osd.2 on smithi178 2024-02-14T17:39:28.152 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:27 smithi178 ceph-mon[39811]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:28.152 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:27 smithi178 ceph-mon[30321]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:28.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:27 smithi184 ceph-mon[33536]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:30.144 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:29 smithi178 ceph-mon[30321]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:30.144 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:29 smithi178 ceph-mon[39811]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:30.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:29 smithi184 ceph-mon[33536]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:31.707 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:31 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:31.708 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:31 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:31.708 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:31 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:31.708 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:31 smithi178 ceph-mon[30321]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:31.708 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:31 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:31.708 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:31 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:31.708 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:31 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:31.708 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:31 smithi178 ceph-mon[39811]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:31 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:31 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:31 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:31 smithi184 ceph-mon[33536]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:33 smithi178 ceph-mon[30321]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:33 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:33 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:33 smithi178 ceph-mon[39811]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:33 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:33 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:34.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:33 smithi184 ceph-mon[33536]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:34.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:33 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:33 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:34.477 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 2 on host 'smithi178' 2024-02-14T17:39:34.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:34.474+0000 7f747effd700 1 -- 172.21.15.178:0/338029410 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3187332026) 0x7f74880fccb0 con 0x7f747406ea00 2024-02-14T17:39:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:34.476+0000 7f748fc1f700 1 -- 172.21.15.178:0/338029410 >> v1:172.21.15.178:6800/1500761607 conn(0x7f747406ea00 legacy=0x7f7474070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:34.476+0000 7f748fc1f700 1 -- 172.21.15.178:0/338029410 >> v1:172.21.15.178:6789/0 conn(0x7f7488108ae0 legacy=0x7f74881a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:34.476+0000 7f748fc1f700 1 -- 172.21.15.178:0/338029410 shutdown_connections 2024-02-14T17:39:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:34.477+0000 7f748fc1f700 1 -- 172.21.15.178:0/338029410 >> 172.21.15.178:0/338029410 conn(0x7f74880fb6e0 msgr2=0x7f7488100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:34.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:34.477+0000 7f748fc1f700 1 -- 172.21.15.178:0/338029410 shutdown_connections 2024-02-14T17:39:34.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:34.477+0000 7f748fc1f700 1 -- 172.21.15.178:0/338029410 wait complete. 2024-02-14T17:39:35.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[39811]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:39:35.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:35.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:35.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:35.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:35.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:35.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[30321]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:39:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:34 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:35.108 DEBUG:teuthology.orchestra.run.smithi178:osd.2> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.2.service 2024-02-14T17:39:35.111 INFO:tasks.cephadm:Deploying osd.3 on smithi178 with /dev/vg_nvme/lv_1... 2024-02-14T17:39:35.111 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-14T17:39:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:34 smithi184 ceph-mon[33536]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:39:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:34 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:34 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:34 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:34 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:34 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:34 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:36.047 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:35 smithi178 ceph-mon[39811]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:36.047 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:35 smithi178 ceph-mon[39811]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T17:39:36.047 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:35 smithi178 ceph-mon[39811]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T17:39:36.047 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:35 smithi178 ceph-mon[39811]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:39:36.047 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:35 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:36.048 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:35 smithi178 ceph-mon[30321]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:36.048 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:35 smithi178 ceph-mon[30321]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T17:39:36.048 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:35 smithi178 ceph-mon[30321]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T17:39:36.048 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:35 smithi178 ceph-mon[30321]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:39:36.048 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:35 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:35 smithi184 ceph-mon[33536]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:35 smithi184 ceph-mon[33536]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T17:39:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:35 smithi184 ceph-mon[33536]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T17:39:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:35 smithi184 ceph-mon[33536]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:39:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:35 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:36.330 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2[56763]: 2024-02-14T17:39:36.071+0000 7f0a4d64e700 -1 osd.2 0 waiting for initial osdmap 2024-02-14T17:39:36.330 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2[56763]: 2024-02-14T17:39:36.073+0000 7f0a45439700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:39:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:39:36.942 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[30321]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:39:36.942 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[30321]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T17:39:36.942 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:36.943 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:36.943 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[30321]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' 2024-02-14T17:39:36.943 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:36.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[39811]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:39:36.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[39811]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T17:39:36.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:36.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:36.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[39811]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' 2024-02-14T17:39:36.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:36 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:36 smithi184 ceph-mon[33536]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:39:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:36 smithi184 ceph-mon[33536]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T17:39:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:36 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:36 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:36 smithi184 ceph-mon[33536]: from='osd.2 v1:172.21.15.178:6809/479756253' entity='osd.2' 2024-02-14T17:39:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:36 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:37.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[39811]: purged_snaps scrub starts 2024-02-14T17:39:37.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[39811]: purged_snaps scrub ok 2024-02-14T17:39:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[39811]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[39811]: osd.2 v1:172.21.15.178:6809/479756253 boot 2024-02-14T17:39:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[39811]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T17:39:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[30321]: purged_snaps scrub starts 2024-02-14T17:39:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[30321]: purged_snaps scrub ok 2024-02-14T17:39:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[30321]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[30321]: osd.2 v1:172.21.15.178:6809/479756253 boot 2024-02-14T17:39:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[30321]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T17:39:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:37 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:37 smithi184 ceph-mon[33536]: purged_snaps scrub starts 2024-02-14T17:39:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:37 smithi184 ceph-mon[33536]: purged_snaps scrub ok 2024-02-14T17:39:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:37 smithi184 ceph-mon[33536]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:39:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:37 smithi184 ceph-mon[33536]: osd.2 v1:172.21.15.178:6809/479756253 boot 2024-02-14T17:39:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:37 smithi184 ceph-mon[33536]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T17:39:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:37 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:39:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T17:39:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: Detected new or changed devices on smithi178 2024-02-14T17:39:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi178 to 5354M 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: pgmap v78: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: Detected new or changed devices on smithi178 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:39.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:39.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:39.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi178 to 5354M 2024-02-14T17:39:39.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:39.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:39.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: pgmap v78: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:39.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:39:39.424 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-14T17:39:39.424 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-14T17:39:39.424 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-02-14T17:39:39.424 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-02-14T17:39:39.424 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00876559 s, 1.2 GB/s 2024-02-14T17:39:39.424 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-02-14T17:39:39.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: Detected new or changed devices on smithi178 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi178 to 5354M 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: pgmap v78: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:39:40.114 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi178:vg_nvme/lv_1 2024-02-14T17:39:40.375 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:39:40.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:40 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T17:39:40.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:40 smithi184 ceph-mon[33536]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T17:39:40.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:40 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:39:40.499 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:40 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T17:39:40.499 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:40 smithi178 ceph-mon[30321]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T17:39:40.499 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:40 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:39:40.500 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:40 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T17:39:40.500 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:40 smithi178 ceph-mon[39811]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T17:39:40.500 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:40 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:39:40.782 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 17:39:40 smithi178 sudo[61195]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:39:40.782 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 17:39:40 smithi178 sudo[61195]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:40.782 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 17:39:40 smithi178 sudo[61195]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:40.782 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 17:39:40 smithi178 sudo[61195]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:41.073 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 17:39:40 smithi178 sudo[61230]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:39:41.074 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 17:39:40 smithi178 sudo[61230]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:41.074 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 17:39:40 smithi178 sudo[61230]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:41.074 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 17:39:40 smithi178 sudo[61230]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:41.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:41 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:39:41.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:41 smithi178 ceph-mon[30321]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T17:39:41.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:41 smithi178 ceph-mon[30321]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:41.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:41 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:41.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:41 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:39:41.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:41 smithi178 ceph-mon[39811]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T17:39:41.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:41 smithi178 ceph-mon[39811]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:41.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:41 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:41.334 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61238]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:39:41.334 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61238]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:41.334 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61238]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:41.334 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61238]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:41.602 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61260]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:39:41.603 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61260]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:41.603 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61260]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:41.603 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61260]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:41.603 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61250]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:39:41.603 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61250]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:41.603 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61250]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:41.603 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61250]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:41 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:39:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:41 smithi184 ceph-mon[33536]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T17:39:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:41 smithi184 ceph-mon[33536]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:41 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:41.857 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61289]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:39:42.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.061+0000 7f3595e25700 1 -- 172.21.15.178:0/2475608521 >> v1:172.21.15.178:6790/0 conn(0x7f3590104d90 legacy=0x7f35901071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:42.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.061+0000 7f3595e25700 1 -- 172.21.15.178:0/2475608521 shutdown_connections 2024-02-14T17:39:42.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.061+0000 7f3595e25700 1 -- 172.21.15.178:0/2475608521 >> 172.21.15.178:0/2475608521 conn(0x7f35900fbf00 msgr2=0x7f35900fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:42.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.061+0000 7f3595e25700 1 -- 172.21.15.178:0/2475608521 shutdown_connections 2024-02-14T17:39:42.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.061+0000 7f3595e25700 1 -- 172.21.15.178:0/2475608521 wait complete. 2024-02-14T17:39:42.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.062+0000 7f3595e25700 1 Processor -- start 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.062+0000 7f3595e25700 1 -- start start 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.062+0000 7f3595e25700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3590197990 con 0x7f3590104d90 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.062+0000 7f3595e25700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f35901a3180 con 0x7f3590108ad0 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.062+0000 7f3595e25700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f35901a4320 con 0x7f3590100960 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.062+0000 7f358f7fe700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f3590100960 0x7f3590196e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:46562/0 (socket says 172.21.15.178:46562) 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.062+0000 7f358f7fe700 1 -- 172.21.15.178:0/1299963662 learned_addr learned my addr 172.21.15.178:0/1299963662 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770789313 0 0) 0x7f35901a4320 con 0x7f3590100960 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35780037f0 con 0x7f3590100960 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086902658 0 0) 0x7f35901a3180 con 0x7f3590108ad0 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35901a4320 con 0x7f3590108ad0 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3143907726 0 0) 0x7f35780037f0 con 0x7f3590100960 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35901a3180 con 0x7f3590100960 2024-02-14T17:39:42.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f3580001870 con 0x7f3590100960 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2572813891 0 0) 0x7f35901a4320 con 0x7f3590108ad0 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35780037f0 con 0x7f3590108ad0 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f3588001c30 con 0x7f3590108ad0 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2155723118 0 0) 0x7f35901a3180 con 0x7f3590100960 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 >> v1:172.21.15.184:6789/0 conn(0x7f3590108ad0 legacy=0x7f35901a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 >> v1:172.21.15.178:6789/0 conn(0x7f3590104d90 legacy=0x7f359019d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35901a54c0 con 0x7f3590100960 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f3595e25700 1 -- 172.21.15.178:0/1299963662 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35901a21d0 con 0x7f3590100960 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f3595e25700 1 -- 172.21.15.178:0/1299963662 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f35901a26d0 con 0x7f3590100960 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f3580003b30 con 0x7f3590100960 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.063+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f3580005c00 con 0x7f3590100960 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.064+0000 7f3595e25700 1 -- 172.21.15.178:0/1299963662 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357c005120 con 0x7f3590100960 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.065+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 14) v1 ==== 91316+0+0 (unknown 3490801675 0 0) 0x7f35800036a0 con 0x7f3590100960 2024-02-14T17:39:42.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.065+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(22..22 src has 1..22) v4 ==== 2511+0+0 (unknown 4129659520 0 0) 0x7f35800877c0 con 0x7f3590100960 2024-02-14T17:39:42.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.073+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f3580051e90 con 0x7f3590100960 2024-02-14T17:39:42.109 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61289]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:42.109 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61289]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:42.109 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:39:41 smithi178 sudo[61289]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.285+0000 7f3595e25700 1 -- 172.21.15.178:0/1299963662 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f357c000b90 con 0x7f357806e990 2024-02-14T17:39:42.370 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 sudo[61326]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:39:42.370 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 sudo[61326]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:42.370 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 sudo[61326]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:42.370 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 sudo[61326]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:42.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:39:42.381+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mon.2 v1:172.21.15.178:6790/0 10 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7f3580051940 con 0x7f3590100960 2024-02-14T17:39:42.637 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 ceph-mon[33536]: osdmap e22: 3 total, 3 up, 3 in 2024-02-14T17:39:42.637 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:39:42.637 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:39:42.637 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:39:42.637 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 sudo[36336]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:39:42.637 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 sudo[36336]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:42.638 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 sudo[36336]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:42.638 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 sudo[36336]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:42.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:42 smithi178 ceph-mon[39811]: osdmap e22: 3 total, 3 up, 3 in 2024-02-14T17:39:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:42 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:39:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:42 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:39:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:42 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:39:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 sudo[61331]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:39:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 sudo[61331]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 sudo[61331]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 sudo[61331]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 ceph-mon[30321]: osdmap e22: 3 total, 3 up, 3 in 2024-02-14T17:39:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:39:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:39:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:42 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:39:42.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 sudo[36339]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:39:42.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 sudo[36339]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:42.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 sudo[36339]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:42.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:42 smithi184 sudo[36339]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 sudo[61473]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:39:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 sudo[61473]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 sudo[61473]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 sudo[61473]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='client.34199 v1:172.21.15.178:0/1299963662' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:39:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:39:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:39:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:39:43.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 sudo[61505]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:39:43.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 sudo[61505]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:39:43.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 sudo[61505]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:39:43.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 sudo[61505]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:39:43.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:39:43.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='client.34199 v1:172.21.15.178:0/1299963662' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:39:43.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:39:43.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:39:43.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:39:43.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='client.34199 v1:172.21.15.178:0/1299963662' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:43.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:39:43.734 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:39:43.734 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:39:44.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:44 smithi178 ceph-mon[30321]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:39:44.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:44 smithi178 ceph-mon[30321]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:39:44.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:44 smithi178 ceph-mon[39811]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:39:44.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:44 smithi178 ceph-mon[39811]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:39:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:44 smithi184 ceph-mon[33536]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:39:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:44 smithi184 ceph-mon[33536]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:39:45.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:45 smithi184 ceph-mon[33536]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:45.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2900467968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "058ca4e0-7fc0-4b14-93ef-fd9fa2fecd0e"}]: dispatch 2024-02-14T17:39:45.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2900467968' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "058ca4e0-7fc0-4b14-93ef-fd9fa2fecd0e"}]': finished 2024-02-14T17:39:45.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:45 smithi184 ceph-mon[33536]: osdmap e23: 4 total, 3 up, 4 in 2024-02-14T17:39:45.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:39:45.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:45 smithi178 ceph-mon[30321]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:45.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2900467968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "058ca4e0-7fc0-4b14-93ef-fd9fa2fecd0e"}]: dispatch 2024-02-14T17:39:45.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2900467968' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "058ca4e0-7fc0-4b14-93ef-fd9fa2fecd0e"}]': finished 2024-02-14T17:39:45.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:45 smithi178 ceph-mon[30321]: osdmap e23: 4 total, 3 up, 4 in 2024-02-14T17:39:45.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:39:45.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:45 smithi178 ceph-mon[39811]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:45.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2900467968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "058ca4e0-7fc0-4b14-93ef-fd9fa2fecd0e"}]: dispatch 2024-02-14T17:39:45.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2900467968' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "058ca4e0-7fc0-4b14-93ef-fd9fa2fecd0e"}]': finished 2024-02-14T17:39:45.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:45 smithi178 ceph-mon[39811]: osdmap e23: 4 total, 3 up, 4 in 2024-02-14T17:39:45.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:39:46.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3214944110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:39:46.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3214944110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:39:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3214944110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:39:47.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:47 smithi184 ceph-mon[33536]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:47 smithi178 ceph-mon[30321]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:47 smithi178 ceph-mon[39811]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:39:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:49 smithi178 ceph-mon[30321]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-14T17:39:50.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:49 smithi178 ceph-mon[39811]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-14T17:39:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:49 smithi184 ceph-mon[33536]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-14T17:39:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:51 smithi178 ceph-mon[39811]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-14T17:39:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:51 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-14T17:39:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:51 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:51 smithi178 ceph-mon[30321]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-14T17:39:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-14T17:39:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:52.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:51 smithi184 ceph-mon[33536]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-14T17:39:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:51 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-14T17:39:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:51 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:52 smithi178 ceph-mon[30321]: Deploying daemon osd.3 on smithi178 2024-02-14T17:39:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:52 smithi178 ceph-mon[39811]: Deploying daemon osd.3 on smithi178 2024-02-14T17:39:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:52 smithi184 ceph-mon[33536]: Deploying daemon osd.3 on smithi178 2024-02-14T17:39:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:53 smithi178 ceph-mon[30321]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:39:54.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:53 smithi178 ceph-mon[39811]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:39:54.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:53 smithi184 ceph-mon[33536]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:39:56.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:55 smithi178 ceph-mon[30321]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:39:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:55 smithi178 ceph-mon[39811]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:39:56.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:55 smithi184 ceph-mon[33536]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:39:57.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:57 smithi178 ceph-mon[39811]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 68 KiB/s wr, 4 op/s 2024-02-14T17:39:57.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:57 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:57.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:57 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:57.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:57 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:57.978 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:57 smithi178 ceph-mon[30321]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 68 KiB/s wr, 4 op/s 2024-02-14T17:39:57.978 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:57 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:57.978 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:57 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:57.979 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:57 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:58.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:57 smithi184 ceph-mon[33536]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 68 KiB/s wr, 4 op/s 2024-02-14T17:39:58.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:58.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:39:58.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:57 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:00.074 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:39:59 smithi178 ceph-mon[39811]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-14T17:40:00.075 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:39:59 smithi178 ceph-mon[30321]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-14T17:40:00.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:39:59 smithi184 ceph-mon[33536]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-14T17:40:00.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[39811]: overall HEALTH_OK 2024-02-14T17:40:00.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[39811]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T17:40:00.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:00.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:00.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:00.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:00.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:00.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[30321]: overall HEALTH_OK 2024-02-14T17:40:00.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[30321]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T17:40:00.978 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:00.978 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:00.978 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:00.978 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:00.978 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:00 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:01.030 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 3 on host 'smithi178' 2024-02-14T17:40:01.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:40:01.028+0000 7f358cff9700 1 -- 172.21.15.178:0/1299963662 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3790552334) 0x7f357c000b90 con 0x7f357806e990 2024-02-14T17:40:01.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:40:01.030+0000 7f3595e25700 1 -- 172.21.15.178:0/1299963662 >> v1:172.21.15.178:6800/1500761607 conn(0x7f357806e990 legacy=0x7f3578070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:01.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:40:01.030+0000 7f3595e25700 1 -- 172.21.15.178:0/1299963662 >> v1:172.21.15.178:6790/0 conn(0x7f3590100960 legacy=0x7f3590196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:01.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:40:01.030+0000 7f3595e25700 1 -- 172.21.15.178:0/1299963662 shutdown_connections 2024-02-14T17:40:01.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:40:01.030+0000 7f3595e25700 1 -- 172.21.15.178:0/1299963662 >> 172.21.15.178:0/1299963662 conn(0x7f35900fbf00 msgr2=0x7f359006a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:01.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:40:01.030+0000 7f3595e25700 1 -- 172.21.15.178:0/1299963662 shutdown_connections 2024-02-14T17:40:01.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:40:01.030+0000 7f3595e25700 1 -- 172.21.15.178:0/1299963662 wait complete. 2024-02-14T17:40:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:00 smithi184 ceph-mon[33536]: overall HEALTH_OK 2024-02-14T17:40:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:00 smithi184 ceph-mon[33536]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T17:40:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:00 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:00 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:00 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:00 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:00 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:01.658 DEBUG:teuthology.orchestra.run.smithi178:osd.3> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.3.service 2024-02-14T17:40:01.661 INFO:tasks.cephadm:Deploying osd.4 on smithi184 with /dev/vg_nvme/lv_4... 2024-02-14T17:40:01.661 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-14T17:40:01.904 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:40:01.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[39811]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:40:01.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[39811]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-14T17:40:01.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[39811]: osdmap e24: 4 total, 3 up, 4 in 2024-02-14T17:40:01.962 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[39811]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:40:01.962 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:01.962 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:01.962 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:01.962 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:01.962 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[39811]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:40:01.962 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[39811]: osdmap e25: 4 total, 3 up, 4 in 2024-02-14T17:40:01.962 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[30321]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:40:01.962 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[30321]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-14T17:40:01.962 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[30321]: osdmap e24: 4 total, 3 up, 4 in 2024-02-14T17:40:01.963 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[30321]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:40:01.963 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:01.963 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:01.963 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:01.963 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:01.963 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[30321]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:40:01.963 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:01 smithi178 ceph-mon[30321]: osdmap e25: 4 total, 3 up, 4 in 2024-02-14T17:40:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:01 smithi184 ceph-mon[33536]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:40:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:01 smithi184 ceph-mon[33536]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-14T17:40:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:01 smithi184 ceph-mon[33536]: osdmap e24: 4 total, 3 up, 4 in 2024-02-14T17:40:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:01 smithi184 ceph-mon[33536]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-02-14T17:40:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:01 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:01 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:01 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:01 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:01 smithi184 ceph-mon[33536]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-02-14T17:40:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:01 smithi184 ceph-mon[33536]: osdmap e25: 4 total, 3 up, 4 in 2024-02-14T17:40:02.219 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 17:40:02 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3[63393]: 2024-02-14T17:40:02.077+0000 7f74043e8700 -1 osd.3 0 waiting for initial osdmap 2024-02-14T17:40:02.219 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 17:40:02 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3[63393]: 2024-02-14T17:40:02.080+0000 7f73fc1d3700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:40:02.982 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:02.982 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:02.982 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:02 smithi184 ceph-mon[33536]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' 2024-02-14T17:40:02.982 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:02 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:03.154 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:02 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:03.155 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:02 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:03.155 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:02 smithi178 ceph-mon[39811]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' 2024-02-14T17:40:03.155 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:02 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:03.155 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:03.155 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:03.155 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:02 smithi178 ceph-mon[30321]: from='osd.3 v1:172.21.15.178:6813/731412783' entity='osd.3' 2024-02-14T17:40:03.155 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:02 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:03.884 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-14T17:40:03.884 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-14T17:40:03.884 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-02-14T17:40:03.885 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-02-14T17:40:03.885 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114512 s, 916 MB/s 2024-02-14T17:40:03.885 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-02-14T17:40:04.195 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:03 smithi184 ceph-mon[33536]: purged_snaps scrub starts 2024-02-14T17:40:04.195 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:03 smithi184 ceph-mon[33536]: purged_snaps scrub ok 2024-02-14T17:40:04.196 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:03 smithi184 ceph-mon[33536]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:40:04.196 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:03 smithi184 ceph-mon[33536]: osd.3 v1:172.21.15.178:6813/731412783 boot 2024-02-14T17:40:04.196 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:03 smithi184 ceph-mon[33536]: osdmap e26: 4 total, 4 up, 4 in 2024-02-14T17:40:04.196 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:03 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:04.296 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[30321]: purged_snaps scrub starts 2024-02-14T17:40:04.296 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[30321]: purged_snaps scrub ok 2024-02-14T17:40:04.296 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[30321]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:40:04.299 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[30321]: osd.3 v1:172.21.15.178:6813/731412783 boot 2024-02-14T17:40:04.299 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[30321]: osdmap e26: 4 total, 4 up, 4 in 2024-02-14T17:40:04.299 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:04.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[39811]: purged_snaps scrub starts 2024-02-14T17:40:04.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[39811]: purged_snaps scrub ok 2024-02-14T17:40:04.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[39811]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:40:04.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[39811]: osd.3 v1:172.21.15.178:6813/731412783 boot 2024-02-14T17:40:04.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[39811]: osdmap e26: 4 total, 4 up, 4 in 2024-02-14T17:40:04.300 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:03 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:40:04.505 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_4 2024-02-14T17:40:04.738 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:40:05.119 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: osdmap e27: 4 total, 4 up, 4 in 2024-02-14T17:40:05.119 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: Detected new or changed devices on smithi178 2024-02-14T17:40:05.119 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.119 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi178 to 4016M 2024-02-14T17:40:05.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:05.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:05.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:05 smithi184 ceph-mon[33536]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:05.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: osdmap e27: 4 total, 4 up, 4 in 2024-02-14T17:40:05.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: Detected new or changed devices on smithi178 2024-02-14T17:40:05.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi178 to 4016M 2024-02-14T17:40:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[30321]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:05.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: osdmap e27: 4 total, 4 up, 4 in 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: Detected new or changed devices on smithi178 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi178 to 4016M 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:05.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:05 smithi178 ceph-mon[39811]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:06.411 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.409+0000 7f49a8bc6700 1 -- 172.21.15.184:0/897101114 >> v1:172.21.15.184:6789/0 conn(0x7f49a4101f80 legacy=0x7f49a41023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:06.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.409+0000 7f49a8bc6700 1 -- 172.21.15.184:0/897101114 shutdown_connections 2024-02-14T17:40:06.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.409+0000 7f49a8bc6700 1 -- 172.21.15.184:0/897101114 >> 172.21.15.184:0/897101114 conn(0x7f49a40fb690 msgr2=0x7f49a40fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:06.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.409+0000 7f49a8bc6700 1 -- 172.21.15.184:0/897101114 shutdown_connections 2024-02-14T17:40:06.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.409+0000 7f49a8bc6700 1 -- 172.21.15.184:0/897101114 wait complete. 2024-02-14T17:40:06.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.409+0000 7f49a8bc6700 1 Processor -- start 2024-02-14T17:40:06.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f49a8bc6700 1 -- start start 2024-02-14T17:40:06.414 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f49a8bc6700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f49a410d840 con 0x7f49a4108ad0 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f49a8bc6700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f49a41969f0 con 0x7f49a4104e20 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f49a8bc6700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f49a41a3190 con 0x7f49a4101f80 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f4999d9b700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f49a4104e20 0x7f49a419c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:43416/0 (socket says 172.21.15.184:43416) 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f4999d9b700 1 -- 172.21.15.184:0/3665794803 learned_addr learned my addr 172.21.15.184:0/3665794803 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 645152136 0 0) 0x7f49a41969f0 con 0x7f49a4104e20 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49900035b0 con 0x7f49a4104e20 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082887872 0 0) 0x7f49a41a3190 con 0x7f49a4101f80 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a41969f0 con 0x7f49a4101f80 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543356231 0 0) 0x7f49a410d840 con 0x7f49a4108ad0 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a41a3190 con 0x7f49a4108ad0 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1249504138 0 0) 0x7f49900035b0 con 0x7f49a4104e20 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49a410d840 con 0x7f49a4104e20 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f498c001870 con 0x7f49a4104e20 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2935096267 0 0) 0x7f49a41969f0 con 0x7f49a4101f80 2024-02-14T17:40:06.415 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.410+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49900035b0 con 0x7f49a4101f80 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3681907888 0 0) 0x7f49a41a3190 con 0x7f49a4108ad0 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49a41969f0 con 0x7f49a4108ad0 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f4994004830 con 0x7f49a4101f80 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f499c001770 con 0x7f49a4108ad0 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1462982344 0 0) 0x7f49a410d840 con 0x7f49a4104e20 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 >> v1:172.21.15.178:6790/0 conn(0x7f49a4101f80 legacy=0x7f49a4195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 >> v1:172.21.15.178:6789/0 conn(0x7f49a4108ad0 legacy=0x7f49a419fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a41a4330 con 0x7f49a4104e20 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f49a8bc6700 1 -- 172.21.15.184:0/3665794803 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a41a3380 con 0x7f49a4104e20 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f49a8bc6700 1 -- 172.21.15.184:0/3665794803 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a41a3880 con 0x7f49a4104e20 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f498c003b30 con 0x7f49a4104e20 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.411+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f498c005c30 con 0x7f49a4104e20 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.412+0000 7f49a8bc6700 1 -- 172.21.15.184:0/3665794803 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4984005150 con 0x7f49a4104e20 2024-02-14T17:40:06.416 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.412+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7f498c01c370 con 0x7f49a4104e20 2024-02-14T17:40:06.417 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.413+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(27..27 src has 1..27) v4 ==== 2803+0+0 (unknown 1885913157 0 0) 0x7f498c087b50 con 0x7f49a4104e20 2024-02-14T17:40:06.419 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.417+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f498c052010 con 0x7f49a4104e20 2024-02-14T17:40:06.618 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:06.616+0000 7f49a8bc6700 1 -- 172.21.15.184:0/3665794803 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4984000bc0 con 0x7f499006e970 2024-02-14T17:40:06.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:06 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:40:06.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:06 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:40:06.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:06 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:07.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:06 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:40:07.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:06 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:40:07.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:06 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:06 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:40:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:06 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:40:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:06 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:07.844 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:07 smithi184 ceph-mon[33536]: from='client.24235 v1:172.21.15.184:0/3665794803' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:07.844 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:07 smithi184 ceph-mon[33536]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:07 smithi178 ceph-mon[39811]: from='client.24235 v1:172.21.15.184:0/3665794803' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:07 smithi178 ceph-mon[39811]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:07 smithi178 ceph-mon[30321]: from='client.24235 v1:172.21.15.184:0/3665794803' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:07 smithi178 ceph-mon[30321]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:10.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[30321]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:10.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/1091571187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ea5121-5d8f-48ae-84cd-ba34c8efa7af"}]: dispatch 2024-02-14T17:40:10.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[30321]: from='client.24241 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ea5121-5d8f-48ae-84cd-ba34c8efa7af"}]: dispatch 2024-02-14T17:40:10.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[30321]: from='client.24241 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0ea5121-5d8f-48ae-84cd-ba34c8efa7af"}]': finished 2024-02-14T17:40:10.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[30321]: osdmap e28: 5 total, 4 up, 5 in 2024-02-14T17:40:10.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:10.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[39811]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:10.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/1091571187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ea5121-5d8f-48ae-84cd-ba34c8efa7af"}]: dispatch 2024-02-14T17:40:10.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[39811]: from='client.24241 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ea5121-5d8f-48ae-84cd-ba34c8efa7af"}]: dispatch 2024-02-14T17:40:10.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[39811]: from='client.24241 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0ea5121-5d8f-48ae-84cd-ba34c8efa7af"}]': finished 2024-02-14T17:40:10.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[39811]: osdmap e28: 5 total, 4 up, 5 in 2024-02-14T17:40:10.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:09 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:09 smithi184 ceph-mon[33536]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/1091571187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ea5121-5d8f-48ae-84cd-ba34c8efa7af"}]: dispatch 2024-02-14T17:40:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:09 smithi184 ceph-mon[33536]: from='client.24241 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0ea5121-5d8f-48ae-84cd-ba34c8efa7af"}]: dispatch 2024-02-14T17:40:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:09 smithi184 ceph-mon[33536]: from='client.24241 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0ea5121-5d8f-48ae-84cd-ba34c8efa7af"}]': finished 2024-02-14T17:40:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:09 smithi184 ceph-mon[33536]: osdmap e28: 5 total, 4 up, 5 in 2024-02-14T17:40:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:09 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/451165043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:40:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/451165043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:40:11.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/451165043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:40:12.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:11 smithi178 ceph-mon[39811]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:12.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:11 smithi178 ceph-mon[30321]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:12.092 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:11 smithi184 ceph-mon[33536]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:13 smithi184 ceph-mon[33536]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:13 smithi178 ceph-mon[30321]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:13 smithi178 ceph-mon[39811]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:15 smithi184 ceph-mon[33536]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:15 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-14T17:40:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:15 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:15 smithi178 ceph-mon[39811]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:15 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-14T17:40:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:15 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:16.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:15 smithi178 ceph-mon[30321]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:16.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:15 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-14T17:40:16.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:15 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:16 smithi178 ceph-mon[30321]: Deploying daemon osd.4 on smithi184 2024-02-14T17:40:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:16 smithi178 ceph-mon[39811]: Deploying daemon osd.4 on smithi184 2024-02-14T17:40:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:16 smithi184 ceph-mon[33536]: Deploying daemon osd.4 on smithi184 2024-02-14T17:40:17.922 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:17 smithi184 ceph-mon[33536]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:17 smithi178 ceph-mon[30321]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:17 smithi178 ceph-mon[39811]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:19 smithi178 ceph-mon[30321]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:20.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:19 smithi178 ceph-mon[39811]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:19 smithi184 ceph-mon[33536]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:21.995 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:21 smithi184 ceph-mon[33536]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:21.995 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:21 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:21.995 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:21 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:21.995 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:21 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:21 smithi178 ceph-mon[39811]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:21 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:21 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:21 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:22.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:21 smithi178 ceph-mon[30321]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:22.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:21 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:22.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:21 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:22.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:21 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[39811]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:24.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[30321]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:23 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:24.140 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:23 smithi184 ceph-mon[33536]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:24.140 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:24.140 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:24.140 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:24.140 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:24.140 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:23 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:24.629 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 4 on host 'smithi184' 2024-02-14T17:40:24.630 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:24.628+0000 7f499bfff700 1 -- 172.21.15.184:0/3665794803 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 966886144) 0x7f4984000bc0 con 0x7f499006e970 2024-02-14T17:40:24.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:24.630+0000 7f49a8bc6700 1 -- 172.21.15.184:0/3665794803 >> v1:172.21.15.178:6800/1500761607 conn(0x7f499006e970 legacy=0x7f4990070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:24.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:24.630+0000 7f49a8bc6700 1 -- 172.21.15.184:0/3665794803 >> v1:172.21.15.184:6789/0 conn(0x7f49a4104e20 legacy=0x7f49a419c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:24.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:24.630+0000 7f49a8bc6700 1 -- 172.21.15.184:0/3665794803 shutdown_connections 2024-02-14T17:40:24.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:24.630+0000 7f49a8bc6700 1 -- 172.21.15.184:0/3665794803 >> 172.21.15.184:0/3665794803 conn(0x7f49a40fb690 msgr2=0x7f49a4107210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:24.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:24.630+0000 7f49a8bc6700 1 -- 172.21.15.184:0/3665794803 shutdown_connections 2024-02-14T17:40:24.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:24.630+0000 7f49a8bc6700 1 -- 172.21.15.184:0/3665794803 wait complete. 2024-02-14T17:40:25.204 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:24 smithi184 ceph-mon[33536]: from='osd.4 v1:172.21.15.184:6800/3570504556' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:40:25.204 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:24 smithi184 ceph-mon[33536]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:40:25.204 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:24 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:25.204 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:24 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:25.204 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:24 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:25.232 DEBUG:teuthology.orchestra.run.smithi184:osd.4> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.4.service 2024-02-14T17:40:25.234 INFO:tasks.cephadm:Deploying osd.5 on smithi184 with /dev/vg_nvme/lv_3... 2024-02-14T17:40:25.235 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-14T17:40:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:24 smithi178 ceph-mon[39811]: from='osd.4 v1:172.21.15.184:6800/3570504556' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:40:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:24 smithi178 ceph-mon[39811]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:40:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:24 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:24 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:24 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:25.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:24 smithi178 ceph-mon[30321]: from='osd.4 v1:172.21.15.184:6800/3570504556' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:40:25.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:24 smithi178 ceph-mon[30321]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:40:25.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:24 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:25.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:24 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:25.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:24 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:26.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:25 smithi184 ceph-mon[33536]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:26.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:25 smithi184 ceph-mon[33536]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-14T17:40:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:25 smithi184 ceph-mon[33536]: osdmap e29: 5 total, 4 up, 5 in 2024-02-14T17:40:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:25 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:25 smithi184 ceph-mon[33536]: from='osd.4 v1:172.21.15.184:6800/3570504556' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:25 smithi184 ceph-mon[33536]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:26.280 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:40:26.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[30321]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:26.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[30321]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-14T17:40:26.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[30321]: osdmap e29: 5 total, 4 up, 5 in 2024-02-14T17:40:26.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:26.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[30321]: from='osd.4 v1:172.21.15.184:6800/3570504556' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:26.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[30321]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:26.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[39811]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:26.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[39811]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-14T17:40:26.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[39811]: osdmap e29: 5 total, 4 up, 5 in 2024-02-14T17:40:26.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:26.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[39811]: from='osd.4 v1:172.21.15.184:6800/3570504556' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:26.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:25 smithi178 ceph-mon[39811]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:26.535 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 17:40:26 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T17:40:26.279+0000 7f65d8bcd700 -1 osd.4 0 waiting for initial osdmap 2024-02-14T17:40:26.535 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 17:40:26 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T17:40:26.283+0000 7f65d09b8700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:40:27.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:26 smithi184 ceph-mon[33536]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:40:27.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:26 smithi184 ceph-mon[33536]: osdmap e30: 5 total, 4 up, 5 in 2024-02-14T17:40:27.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:26 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:27.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:26 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:27.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:26 smithi184 ceph-mon[33536]: from='osd.4 ' entity='osd.4' 2024-02-14T17:40:27.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:26 smithi184 ceph-mon[33536]: osd.4 v1:172.21.15.184:6800/3570504556 boot 2024-02-14T17:40:27.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:26 smithi184 ceph-mon[33536]: osdmap e31: 5 total, 5 up, 5 in 2024-02-14T17:40:27.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:26 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[30321]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:40:27.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[30321]: osdmap e30: 5 total, 4 up, 5 in 2024-02-14T17:40:27.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:27.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:27.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[30321]: from='osd.4 ' entity='osd.4' 2024-02-14T17:40:27.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[30321]: osd.4 v1:172.21.15.184:6800/3570504556 boot 2024-02-14T17:40:27.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[30321]: osdmap e31: 5 total, 5 up, 5 in 2024-02-14T17:40:27.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:27.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[39811]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:40:27.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[39811]: osdmap e30: 5 total, 4 up, 5 in 2024-02-14T17:40:27.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:27.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:27.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[39811]: from='osd.4 ' entity='osd.4' 2024-02-14T17:40:27.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[39811]: osd.4 v1:172.21.15.184:6800/3570504556 boot 2024-02-14T17:40:27.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[39811]: osdmap e31: 5 total, 5 up, 5 in 2024-02-14T17:40:27.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:26 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:40:27.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:27 smithi184 ceph-mon[33536]: purged_snaps scrub starts 2024-02-14T17:40:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:27 smithi184 ceph-mon[33536]: purged_snaps scrub ok 2024-02-14T17:40:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:27 smithi184 ceph-mon[33536]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:27 smithi184 ceph-mon[33536]: osdmap e32: 5 total, 5 up, 5 in 2024-02-14T17:40:28.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:27 smithi178 ceph-mon[30321]: purged_snaps scrub starts 2024-02-14T17:40:28.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:27 smithi178 ceph-mon[30321]: purged_snaps scrub ok 2024-02-14T17:40:28.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:27 smithi178 ceph-mon[30321]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:28.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:27 smithi178 ceph-mon[30321]: osdmap e32: 5 total, 5 up, 5 in 2024-02-14T17:40:28.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:27 smithi178 ceph-mon[39811]: purged_snaps scrub starts 2024-02-14T17:40:28.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:27 smithi178 ceph-mon[39811]: purged_snaps scrub ok 2024-02-14T17:40:28.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:27 smithi178 ceph-mon[39811]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-02-14T17:40:28.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:27 smithi178 ceph-mon[39811]: osdmap e32: 5 total, 5 up, 5 in 2024-02-14T17:40:29.112 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-14T17:40:29.112 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-14T17:40:29.112 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-02-14T17:40:29.113 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-02-14T17:40:29.113 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0146546 s, 716 MB/s 2024-02-14T17:40:29.113 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-02-14T17:40:29.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:29 smithi184 ceph-mon[33536]: Detected new or changed devices on smithi184 2024-02-14T17:40:29.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:29 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:29 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:29 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:29.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:29 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi184 to 17088M 2024-02-14T17:40:29.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:29 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:29 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:29.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:29 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:29.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:29 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:29 smithi184 ceph-mon[33536]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 519 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:29.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:29 smithi184 ceph-mon[33536]: osdmap e33: 5 total, 5 up, 5 in 2024-02-14T17:40:29.727 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_3 2024-02-14T17:40:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[39811]: Detected new or changed devices on smithi184 2024-02-14T17:40:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi184 to 17088M 2024-02-14T17:40:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[39811]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 519 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[39811]: osdmap e33: 5 total, 5 up, 5 in 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[30321]: Detected new or changed devices on smithi184 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi184 to 17088M 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[30321]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 519 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:29 smithi178 ceph-mon[30321]: osdmap e33: 5 total, 5 up, 5 in 2024-02-14T17:40:29.961 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:40:31.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:30 smithi184 ceph-mon[33536]: osdmap e34: 5 total, 5 up, 5 in 2024-02-14T17:40:31.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:30 smithi178 ceph-mon[30321]: osdmap e34: 5 total, 5 up, 5 in 2024-02-14T17:40:31.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:30 smithi178 ceph-mon[39811]: osdmap e34: 5 total, 5 up, 5 in 2024-02-14T17:40:31.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.571+0000 7f5182be1700 1 -- 172.21.15.184:0/1088540237 >> v1:172.21.15.184:6789/0 conn(0x7f517c108ad0 legacy=0x7f517c10af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:31.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.571+0000 7f5182be1700 1 -- 172.21.15.184:0/1088540237 shutdown_connections 2024-02-14T17:40:31.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.571+0000 7f5182be1700 1 -- 172.21.15.184:0/1088540237 >> 172.21.15.184:0/1088540237 conn(0x7f517c0fbf00 msgr2=0x7f517c0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:31.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.571+0000 7f5182be1700 1 -- 172.21.15.184:0/1088540237 shutdown_connections 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.571+0000 7f5182be1700 1 -- 172.21.15.184:0/1088540237 wait complete. 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.572+0000 7f5182be1700 1 Processor -- start 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.572+0000 7f5182be1700 1 -- start start 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.572+0000 7f5182be1700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f517c06ad40 con 0x7f517c100960 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.572+0000 7f5182be1700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f517c1a2db0 con 0x7f517c104d90 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.572+0000 7f5182be1700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f517c1a3f50 con 0x7f517c108ad0 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.572+0000 7f517bfff700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f517c104d90 0x7f517c19cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:53722/0 (socket says 172.21.15.184:53722) 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.572+0000 7f517bfff700 1 -- 172.21.15.184:0/1980341399 learned_addr learned my addr 172.21.15.184:0/1980341399 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.572+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321434348 0 0) 0x7f517c1a2db0 con 0x7f517c104d90 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f516c0035b0 con 0x7f517c104d90 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267489062 0 0) 0x7f517c06ad40 con 0x7f517c100960 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f517c1a2db0 con 0x7f517c100960 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076088735 0 0) 0x7f517c1a3f50 con 0x7f517c108ad0 2024-02-14T17:40:31.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f517c06ad40 con 0x7f517c108ad0 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1251672850 0 0) 0x7f516c0035b0 con 0x7f517c104d90 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f517c1a3f50 con 0x7f517c104d90 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f5168001910 con 0x7f517c104d90 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1263667340 0 0) 0x7f517c06ad40 con 0x7f517c108ad0 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f516c0035b0 con 0x7f517c108ad0 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2356473587 0 0) 0x7f517c1a2db0 con 0x7f517c100960 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f517c06ad40 con 0x7f517c100960 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f51740047b0 con 0x7f517c108ad0 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f5170001830 con 0x7f517c100960 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1872648318 0 0) 0x7f517c1a3f50 con 0x7f517c104d90 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 >> v1:172.21.15.178:6790/0 conn(0x7f517c108ad0 legacy=0x7f517c1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 >> v1:172.21.15.178:6789/0 conn(0x7f517c100960 legacy=0x7f517c199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c1a5150 con 0x7f517c104d90 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5182be1700 1 -- 172.21.15.184:0/1980341399 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c1a1e00 con 0x7f517c104d90 2024-02-14T17:40:31.577 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f5168003b30 con 0x7f517c104d90 2024-02-14T17:40:31.578 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5182be1700 1 -- 172.21.15.184:0/1980341399 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c1a2300 con 0x7f517c104d90 2024-02-14T17:40:31.578 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.573+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f5168005c80 con 0x7f517c104d90 2024-02-14T17:40:31.578 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.574+0000 7f5182be1700 1 -- 172.21.15.184:0/1980341399 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517c10e2f0 con 0x7f517c104d90 2024-02-14T17:40:31.578 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.575+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7f5168005f40 con 0x7f517c104d90 2024-02-14T17:40:31.578 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.575+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 3205+0+0 (unknown 2408680846 0 0) 0x7f5168087cd0 con 0x7f517c104d90 2024-02-14T17:40:31.583 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.580+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f5168051f10 con 0x7f517c104d90 2024-02-14T17:40:31.781 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:31.780+0000 7f5182be1700 1 -- 172.21.15.184:0/1980341399 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f517c10db60 con 0x7f516c06e9b0 2024-02-14T17:40:32.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:31 smithi184 ceph-mon[33536]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 519 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:31 smithi184 ceph-mon[33536]: osdmap e35: 5 total, 5 up, 5 in 2024-02-14T17:40:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:31 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:40:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:31 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:40:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:31 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:32.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:31 smithi178 ceph-mon[30321]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 519 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:31 smithi178 ceph-mon[30321]: osdmap e35: 5 total, 5 up, 5 in 2024-02-14T17:40:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:31 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:40:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:31 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:40:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:31 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:32.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:31 smithi178 ceph-mon[39811]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 519 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:32.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:31 smithi178 ceph-mon[39811]: osdmap e35: 5 total, 5 up, 5 in 2024-02-14T17:40:32.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:31 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:40:32.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:31 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:40:32.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:31 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:33.064 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:32 smithi184 ceph-mon[33536]: from='client.24262 v1:172.21.15.184:0/1980341399' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:32 smithi178 ceph-mon[30321]: from='client.24262 v1:172.21.15.184:0/1980341399' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:32 smithi178 ceph-mon[39811]: from='client.24262 v1:172.21.15.184:0/1980341399' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:33.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:33 smithi184 ceph-mon[33536]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-14T17:40:34.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:33 smithi178 ceph-mon[39811]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-14T17:40:34.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:33 smithi178 ceph-mon[30321]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-14T17:40:35.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/182215297' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165cded3-2161-4e1c-93f4-719dd6c7cc0e"}]: dispatch 2024-02-14T17:40:35.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:34 smithi184 ceph-mon[33536]: from='client.24268 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165cded3-2161-4e1c-93f4-719dd6c7cc0e"}]: dispatch 2024-02-14T17:40:35.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:34 smithi184 ceph-mon[33536]: from='client.24268 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "165cded3-2161-4e1c-93f4-719dd6c7cc0e"}]': finished 2024-02-14T17:40:35.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:34 smithi184 ceph-mon[33536]: osdmap e36: 6 total, 5 up, 6 in 2024-02-14T17:40:35.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:34 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:35.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/182215297' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165cded3-2161-4e1c-93f4-719dd6c7cc0e"}]: dispatch 2024-02-14T17:40:35.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:34 smithi178 ceph-mon[39811]: from='client.24268 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165cded3-2161-4e1c-93f4-719dd6c7cc0e"}]: dispatch 2024-02-14T17:40:35.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:34 smithi178 ceph-mon[39811]: from='client.24268 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "165cded3-2161-4e1c-93f4-719dd6c7cc0e"}]': finished 2024-02-14T17:40:35.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:34 smithi178 ceph-mon[39811]: osdmap e36: 6 total, 5 up, 6 in 2024-02-14T17:40:35.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:34 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:35.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/182215297' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165cded3-2161-4e1c-93f4-719dd6c7cc0e"}]: dispatch 2024-02-14T17:40:35.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:34 smithi178 ceph-mon[30321]: from='client.24268 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165cded3-2161-4e1c-93f4-719dd6c7cc0e"}]: dispatch 2024-02-14T17:40:35.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:34 smithi178 ceph-mon[30321]: from='client.24268 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "165cded3-2161-4e1c-93f4-719dd6c7cc0e"}]': finished 2024-02-14T17:40:35.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:34 smithi178 ceph-mon[30321]: osdmap e36: 6 total, 5 up, 6 in 2024-02-14T17:40:35.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:34 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:36.144 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:35 smithi184 ceph-mon[33536]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-14T17:40:36.144 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/2949978229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:40:36.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:35 smithi178 ceph-mon[30321]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-14T17:40:36.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/2949978229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:40:36.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:35 smithi178 ceph-mon[39811]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-14T17:40:36.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/2949978229' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:40:38.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:37 smithi184 ceph-mon[33536]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-14T17:40:38.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:37 smithi178 ceph-mon[30321]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-14T17:40:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:37 smithi178 ceph-mon[39811]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-14T17:40:40.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:39 smithi184 ceph-mon[33536]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:40:40.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:39 smithi178 ceph-mon[39811]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:40:40.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:39 smithi178 ceph-mon[30321]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:40:42.135 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:41 smithi178 ceph-mon[39811]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T17:40:42.135 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:41 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-14T17:40:42.135 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:41 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:42.135 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:41 smithi178 ceph-mon[30321]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T17:40:42.135 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:41 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-14T17:40:42.135 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:41 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:42.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:41 smithi184 ceph-mon[33536]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T17:40:42.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:41 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-14T17:40:42.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:41 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:43.150 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:42 smithi178 ceph-mon[39811]: Deploying daemon osd.5 on smithi184 2024-02-14T17:40:43.150 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:42 smithi178 ceph-mon[30321]: Deploying daemon osd.5 on smithi184 2024-02-14T17:40:43.200 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:42 smithi184 ceph-mon[33536]: Deploying daemon osd.5 on smithi184 2024-02-14T17:40:44.146 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:43 smithi184 ceph-mon[33536]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:44.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:43 smithi178 ceph-mon[39811]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:43 smithi178 ceph-mon[30321]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:46.191 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:45 smithi184 ceph-mon[33536]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:46.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:45 smithi178 ceph-mon[39811]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:46.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:45 smithi178 ceph-mon[30321]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:47.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:47 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:47.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:47 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:47.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:47 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:47.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:47 smithi184 ceph-mon[33536]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:47.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:47 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:47 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:47 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:47 smithi178 ceph-mon[39811]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:47 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:47 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:47 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:47 smithi178 ceph-mon[30321]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[30321]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:50.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[30321]: from='osd.5 v1:172.21.15.184:6804/2863982197' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:40:50.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[30321]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:40:50.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[39811]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:50.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:50.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:50.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:50.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:50.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:50.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[39811]: from='osd.5 v1:172.21.15.184:6804/2863982197' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:40:50.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:49 smithi178 ceph-mon[39811]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:40:50.143 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:49 smithi184 ceph-mon[33536]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:50.144 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:50.144 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:50.144 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:50.144 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:50.144 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:49 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:50.144 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:49 smithi184 ceph-mon[33536]: from='osd.5 v1:172.21.15.184:6804/2863982197' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:40:50.144 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:49 smithi184 ceph-mon[33536]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:40:50.144 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 5 on host 'smithi184' 2024-02-14T17:40:50.145 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:50.141+0000 7f5179ffb700 1 -- 172.21.15.184:0/1980341399 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1706323380) 0x7f517c10db60 con 0x7f516c06e9b0 2024-02-14T17:40:50.145 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:50.143+0000 7f5182be1700 1 -- 172.21.15.184:0/1980341399 >> v1:172.21.15.178:6800/1500761607 conn(0x7f516c06e9b0 legacy=0x7f516c070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:50.145 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:50.143+0000 7f5182be1700 1 -- 172.21.15.184:0/1980341399 >> v1:172.21.15.184:6789/0 conn(0x7f517c104d90 legacy=0x7f517c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:50.146 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:50.143+0000 7f5182be1700 1 -- 172.21.15.184:0/1980341399 shutdown_connections 2024-02-14T17:40:50.146 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:50.143+0000 7f5182be1700 1 -- 172.21.15.184:0/1980341399 >> 172.21.15.184:0/1980341399 conn(0x7f517c0fbf00 msgr2=0x7f517c107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:50.146 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:50.143+0000 7f5182be1700 1 -- 172.21.15.184:0/1980341399 shutdown_connections 2024-02-14T17:40:50.146 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:50.143+0000 7f5182be1700 1 -- 172.21.15.184:0/1980341399 wait complete. 2024-02-14T17:40:50.816 DEBUG:teuthology.orchestra.run.smithi184:osd.5> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.5.service 2024-02-14T17:40:50.818 INFO:tasks.cephadm:Deploying osd.6 on smithi184 with /dev/vg_nvme/lv_2... 2024-02-14T17:40:50.818 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-14T17:40:51.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:51 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:51.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:51 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:51.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:51 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:51.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:51 smithi184 ceph-mon[33536]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-14T17:40:51.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:51 smithi184 ceph-mon[33536]: from='osd.5 v1:172.21.15.184:6804/2863982197' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:51.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:51 smithi184 ceph-mon[33536]: osdmap e37: 6 total, 5 up, 6 in 2024-02-14T17:40:51.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:51 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:51.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:51 smithi184 ceph-mon[33536]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:51.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:51 smithi184 ceph-mon[33536]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:51.471 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 17:40:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T17:40:51.237+0000 7f16c4bbd700 -1 osd.5 0 waiting for initial osdmap 2024-02-14T17:40:51.471 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 17:40:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T17:40:51.243+0000 7f16c09b0700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:40:51.525 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:40:51.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:51.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:51.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:51.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[30321]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-14T17:40:51.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[30321]: from='osd.5 v1:172.21.15.184:6804/2863982197' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:51.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[30321]: osdmap e37: 6 total, 5 up, 6 in 2024-02-14T17:40:51.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:51.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[30321]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:51.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[30321]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:51.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:51.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:51.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:51.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[39811]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-14T17:40:51.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[39811]: from='osd.5 v1:172.21.15.184:6804/2863982197' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:51.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[39811]: osdmap e37: 6 total, 5 up, 6 in 2024-02-14T17:40:51.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:51.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[39811]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:40:51.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:51 smithi178 ceph-mon[39811]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:52.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:52 smithi184 ceph-mon[33536]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:40:52.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:52 smithi184 ceph-mon[33536]: osdmap e38: 6 total, 5 up, 6 in 2024-02-14T17:40:52.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:52 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:52.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:52 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:52.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:52 smithi178 ceph-mon[30321]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:40:52.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:52 smithi178 ceph-mon[30321]: osdmap e38: 6 total, 5 up, 6 in 2024-02-14T17:40:52.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:52 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:52.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:52 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:52.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:52 smithi178 ceph-mon[39811]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:40:52.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:52 smithi178 ceph-mon[39811]: osdmap e38: 6 total, 5 up, 6 in 2024-02-14T17:40:52.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:52 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:52.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:52 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:53.507 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:53 smithi184 ceph-mon[33536]: purged_snaps scrub starts 2024-02-14T17:40:53.508 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:53 smithi184 ceph-mon[33536]: purged_snaps scrub ok 2024-02-14T17:40:53.508 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:53 smithi184 ceph-mon[33536]: osd.5 v1:172.21.15.184:6804/2863982197 boot 2024-02-14T17:40:53.508 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:53 smithi184 ceph-mon[33536]: osdmap e39: 6 total, 6 up, 6 in 2024-02-14T17:40:53.508 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:53 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:53.508 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:53 smithi184 ceph-mon[33536]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:53.508 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:53 smithi184 ceph-mon[33536]: osdmap e40: 6 total, 6 up, 6 in 2024-02-14T17:40:53.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[30321]: purged_snaps scrub starts 2024-02-14T17:40:53.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[30321]: purged_snaps scrub ok 2024-02-14T17:40:53.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[30321]: osd.5 v1:172.21.15.184:6804/2863982197 boot 2024-02-14T17:40:53.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[30321]: osdmap e39: 6 total, 6 up, 6 in 2024-02-14T17:40:53.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:53.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[30321]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:53.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[30321]: osdmap e40: 6 total, 6 up, 6 in 2024-02-14T17:40:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[39811]: purged_snaps scrub starts 2024-02-14T17:40:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[39811]: purged_snaps scrub ok 2024-02-14T17:40:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[39811]: osd.5 v1:172.21.15.184:6804/2863982197 boot 2024-02-14T17:40:53.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[39811]: osdmap e39: 6 total, 6 up, 6 in 2024-02-14T17:40:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:40:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[39811]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:40:53.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:53 smithi178 ceph-mon[39811]: osdmap e40: 6 total, 6 up, 6 in 2024-02-14T17:40:54.409 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-14T17:40:54.409 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-14T17:40:54.409 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-02-14T17:40:54.409 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-02-14T17:40:54.409 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113016 s, 928 MB/s 2024-02-14T17:40:54.409 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-02-14T17:40:54.986 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_2 2024-02-14T17:40:55.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:55 smithi184 ceph-mon[33536]: osdmap e41: 6 total, 6 up, 6 in 2024-02-14T17:40:55.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:55 smithi184 ceph-mon[33536]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:40:55.557 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:40:55.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:55 smithi178 ceph-mon[30321]: osdmap e41: 6 total, 6 up, 6 in 2024-02-14T17:40:55.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:55 smithi178 ceph-mon[30321]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:40:55.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:55 smithi178 ceph-mon[39811]: osdmap e41: 6 total, 6 up, 6 in 2024-02-14T17:40:55.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:55 smithi178 ceph-mon[39811]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:40:56.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:56 smithi184 ceph-mon[33536]: osdmap e42: 6 total, 6 up, 6 in 2024-02-14T17:40:56.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:56 smithi184 ceph-mon[33536]: Detected new or changed devices on smithi184 2024-02-14T17:40:56.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:56.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:56.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:56.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:56.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:56 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi184 to 8544M 2024-02-14T17:40:56.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:56.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:56.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:56.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:56.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[39811]: osdmap e42: 6 total, 6 up, 6 in 2024-02-14T17:40:56.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[39811]: Detected new or changed devices on smithi184 2024-02-14T17:40:56.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:56.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:56.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:56.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:56.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi184 to 8544M 2024-02-14T17:40:56.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:56.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:56.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[30321]: osdmap e42: 6 total, 6 up, 6 in 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[30321]: Detected new or changed devices on smithi184 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi184 to 8544M 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:40:57.203 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.199+0000 7f2c92de0700 1 -- 172.21.15.184:0/1087587081 >> v1:172.21.15.184:6789/0 conn(0x7f2c8c105ed0 legacy=0x7f2c8c108320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.199+0000 7f2c92de0700 1 -- 172.21.15.184:0/1087587081 shutdown_connections 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.199+0000 7f2c92de0700 1 -- 172.21.15.184:0/1087587081 >> 172.21.15.184:0/1087587081 conn(0x7f2c8c0fd040 msgr2=0x7f2c8c0ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.199+0000 7f2c92de0700 1 -- 172.21.15.184:0/1087587081 shutdown_connections 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.199+0000 7f2c92de0700 1 -- 172.21.15.184:0/1087587081 wait complete. 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.199+0000 7f2c92de0700 1 Processor -- start 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.199+0000 7f2c92de0700 1 -- start start 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c92de0700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2c8c1a40b0 con 0x7f2c8c101aa0 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c92de0700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2c8c1a52b0 con 0x7f2c8c109c10 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c92de0700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2c8c1a64b0 con 0x7f2c8c105ed0 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c9137d700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f2c8c109c10 0x7f2c8c1a27b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:39102/0 (socket says 172.21.15.184:39102) 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c9137d700 1 -- 172.21.15.184:0/2096232009 learned_addr learned my addr 172.21.15.184:0/2096232009 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:40:57.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347158665 0 0) 0x7f2c8c1a52b0 con 0x7f2c8c109c10 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c740035b0 con 0x7f2c8c109c10 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660407716 0 0) 0x7f2c8c1a40b0 con 0x7f2c8c101aa0 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c8c1a52b0 con 0x7f2c8c101aa0 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915655491 0 0) 0x7f2c8c1a64b0 con 0x7f2c8c105ed0 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c8c1a40b0 con 0x7f2c8c105ed0 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2955333918 0 0) 0x7f2c740035b0 con 0x7f2c8c109c10 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c8c1a64b0 con 0x7f2c8c109c10 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f2c840018f0 con 0x7f2c8c109c10 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3165002647 0 0) 0x7f2c8c1a52b0 con 0x7f2c8c101aa0 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c740035b0 con 0x7f2c8c101aa0 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 905411707 0 0) 0x7f2c8c1a40b0 con 0x7f2c8c105ed0 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c8c1a52b0 con 0x7f2c8c105ed0 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f2c88001b60 con 0x7f2c8c101aa0 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f2c7c0044b0 con 0x7f2c8c105ed0 2024-02-14T17:40:57.205 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1671923430 0 0) 0x7f2c8c1a64b0 con 0x7f2c8c109c10 2024-02-14T17:40:57.206 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 >> v1:172.21.15.178:6790/0 conn(0x7f2c8c105ed0 legacy=0x7f2c8c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:57.206 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 >> v1:172.21.15.178:6789/0 conn(0x7f2c8c101aa0 legacy=0x7f2c8c075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:57.206 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.200+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c1a76b0 con 0x7f2c8c109c10 2024-02-14T17:40:57.206 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.201+0000 7f2c92de0700 1 -- 172.21.15.184:0/2096232009 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8c1a42a0 con 0x7f2c8c109c10 2024-02-14T17:40:57.206 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.201+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f2c84003b30 con 0x7f2c8c109c10 2024-02-14T17:40:57.206 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.201+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f2c84005c90 con 0x7f2c8c109c10 2024-02-14T17:40:57.206 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.201+0000 7f2c92de0700 1 -- 172.21.15.184:0/2096232009 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8c1a47a0 con 0x7f2c8c109c10 2024-02-14T17:40:57.207 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.202+0000 7f2c92de0700 1 -- 172.21.15.184:0/2096232009 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8c068560 con 0x7f2c8c109c10 2024-02-14T17:40:57.207 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.202+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7f2c840036a0 con 0x7f2c8c109c10 2024-02-14T17:40:57.207 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.203+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(43..43 src has 1..43) v4 ==== 3497+0+0 (unknown 1119389314 0 0) 0x7f2c84087fb0 con 0x7f2c8c109c10 2024-02-14T17:40:57.209 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.207+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f2c84052160 con 0x7f2c8c109c10 2024-02-14T17:40:57.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:57 smithi184 ceph-mon[33536]: osdmap e43: 6 total, 6 up, 6 in 2024-02-14T17:40:57.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:57 smithi184 ceph-mon[33536]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:40:57.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:40:57.410+0000 7f2c92de0700 1 -- 172.21.15.184:0/2096232009 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c8c10eec0 con 0x7f2c7406e9c0 2024-02-14T17:40:57.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:57 smithi178 ceph-mon[39811]: osdmap e43: 6 total, 6 up, 6 in 2024-02-14T17:40:57.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:57 smithi178 ceph-mon[39811]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:40:57.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:57 smithi178 ceph-mon[30321]: osdmap e43: 6 total, 6 up, 6 in 2024-02-14T17:40:57.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:57 smithi178 ceph-mon[30321]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:40:58.280 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:58 smithi184 ceph-mon[33536]: from='client.24289 v1:172.21.15.184:0/2096232009' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:58.280 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:58 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:40:58.280 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:58 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:40:58.280 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:58 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:58.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:58 smithi178 ceph-mon[39811]: from='client.24289 v1:172.21.15.184:0/2096232009' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:58.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:58 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:40:58.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:58 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:40:58.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:58 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:58.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:58 smithi178 ceph-mon[30321]: from='client.24289 v1:172.21.15.184:0/2096232009' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:58.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:58 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:40:58.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:58 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:40:58.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:58 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:59.451 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:40:59 smithi184 ceph-mon[33536]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:40:59.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:40:59 smithi178 ceph-mon[39811]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:40:59.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:40:59 smithi178 ceph-mon[30321]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:00.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/1430896495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93c78324-6494-44c1-96b1-4c957ae7b33f"}]: dispatch 2024-02-14T17:41:00.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:00 smithi184 ceph-mon[33536]: from='client.24295 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93c78324-6494-44c1-96b1-4c957ae7b33f"}]: dispatch 2024-02-14T17:41:00.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:00 smithi184 ceph-mon[33536]: from='client.24295 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93c78324-6494-44c1-96b1-4c957ae7b33f"}]': finished 2024-02-14T17:41:00.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:00 smithi184 ceph-mon[33536]: osdmap e44: 7 total, 6 up, 7 in 2024-02-14T17:41:00.473 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:00 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:00.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/1430896495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93c78324-6494-44c1-96b1-4c957ae7b33f"}]: dispatch 2024-02-14T17:41:00.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:00 smithi178 ceph-mon[39811]: from='client.24295 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93c78324-6494-44c1-96b1-4c957ae7b33f"}]: dispatch 2024-02-14T17:41:00.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:00 smithi178 ceph-mon[39811]: from='client.24295 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93c78324-6494-44c1-96b1-4c957ae7b33f"}]': finished 2024-02-14T17:41:00.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:00 smithi178 ceph-mon[39811]: osdmap e44: 7 total, 6 up, 7 in 2024-02-14T17:41:00.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:00 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:00.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/1430896495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93c78324-6494-44c1-96b1-4c957ae7b33f"}]: dispatch 2024-02-14T17:41:00.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:00 smithi178 ceph-mon[30321]: from='client.24295 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93c78324-6494-44c1-96b1-4c957ae7b33f"}]: dispatch 2024-02-14T17:41:00.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:00 smithi178 ceph-mon[30321]: from='client.24295 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93c78324-6494-44c1-96b1-4c957ae7b33f"}]': finished 2024-02-14T17:41:00.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:00 smithi178 ceph-mon[30321]: osdmap e44: 7 total, 6 up, 7 in 2024-02-14T17:41:00.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:00 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:01.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/2862626617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:01.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:01 smithi184 ceph-mon[33536]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:01.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/2862626617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:01.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:01 smithi178 ceph-mon[30321]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:01.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/2862626617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:01.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:01 smithi178 ceph-mon[39811]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:03 smithi178 ceph-mon[39811]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:03 smithi178 ceph-mon[30321]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:04.176 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:03 smithi184 ceph-mon[33536]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:06.077 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:05 smithi184 ceph-mon[33536]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:06.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:05 smithi178 ceph-mon[39811]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:06.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:05 smithi178 ceph-mon[30321]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:06.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:06 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-14T17:41:06.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:06 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:07.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:06 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-14T17:41:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:06 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:07.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:06 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-14T17:41:07.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:06 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:07.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:07 smithi184 ceph-mon[33536]: Deploying daemon osd.6 on smithi184 2024-02-14T17:41:07.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:07 smithi184 ceph-mon[33536]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:08.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:07 smithi178 ceph-mon[30321]: Deploying daemon osd.6 on smithi184 2024-02-14T17:41:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:07 smithi178 ceph-mon[30321]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:07 smithi178 ceph-mon[39811]: Deploying daemon osd.6 on smithi184 2024-02-14T17:41:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:07 smithi178 ceph-mon[39811]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:09.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:09 smithi184 ceph-mon[33536]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:10.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:09 smithi178 ceph-mon[30321]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:09 smithi178 ceph-mon[39811]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:12.094 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:11 smithi184 ceph-mon[33536]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:12.135 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:11 smithi178 ceph-mon[39811]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:12.135 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:11 smithi178 ceph-mon[30321]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:13.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:13 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:13 smithi184 ceph-mon[33536]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:13 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:13 smithi178 ceph-mon[39811]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:13 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:13 smithi178 ceph-mon[30321]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:15 smithi178 ceph-mon[39811]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:15 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:15 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:15 smithi178 ceph-mon[39811]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T17:41:16.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:15 smithi178 ceph-mon[30321]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:16.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:15 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:16.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:15 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:16.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:15 smithi178 ceph-mon[30321]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T17:41:16.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:15 smithi184 ceph-mon[33536]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:16.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:15 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:16.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:15 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:16.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:15 smithi184 ceph-mon[33536]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T17:41:16.179 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:16.177+0000 7f2c81ffb700 1 -- 172.21.15.184:0/2096232009 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 2173329000) 0x7f2c8c10eec0 con 0x7f2c7406e9c0 2024-02-14T17:41:16.179 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 6 on host 'smithi184' 2024-02-14T17:41:16.181 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:16.180+0000 7f2c92de0700 1 -- 172.21.15.184:0/2096232009 >> v1:172.21.15.178:6800/1500761607 conn(0x7f2c7406e9c0 legacy=0x7f2c74070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:16.182 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:16.180+0000 7f2c92de0700 1 -- 172.21.15.184:0/2096232009 >> v1:172.21.15.184:6789/0 conn(0x7f2c8c109c10 legacy=0x7f2c8c1a27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:16.182 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:16.180+0000 7f2c92de0700 1 -- 172.21.15.184:0/2096232009 shutdown_connections 2024-02-14T17:41:16.182 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:16.180+0000 7f2c92de0700 1 -- 172.21.15.184:0/2096232009 >> 172.21.15.184:0/2096232009 conn(0x7f2c8c0fd040 msgr2=0x7f2c8c108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:16.182 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:16.180+0000 7f2c92de0700 1 -- 172.21.15.184:0/2096232009 shutdown_connections 2024-02-14T17:41:16.182 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:16.180+0000 7f2c92de0700 1 -- 172.21.15.184:0/2096232009 wait complete. 2024-02-14T17:41:16.936 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:16.936 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:16.936 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:16.936 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:16.936 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:16.937 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:16.937 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:16 smithi184 ceph-mon[33536]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-14T17:41:16.937 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:16 smithi184 ceph-mon[33536]: osdmap e45: 7 total, 6 up, 7 in 2024-02-14T17:41:16.937 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:16 smithi184 ceph-mon[33536]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:41:16.937 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:16.970 DEBUG:teuthology.orchestra.run.smithi184:osd.6> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.6.service 2024-02-14T17:41:16.972 INFO:tasks.cephadm:Deploying osd.7 on smithi184 with /dev/vg_nvme/lv_1... 2024-02-14T17:41:16.972 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-14T17:41:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[30321]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-14T17:41:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[30321]: osdmap e45: 7 total, 6 up, 7 in 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[30321]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[39811]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[39811]: osdmap e45: 7 total, 6 up, 7 in 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[39811]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:41:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:17.470 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 17:41:17 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T17:41:17.216+0000 7fc2cc597700 -1 osd.6 0 waiting for initial osdmap 2024-02-14T17:41:17.470 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 17:41:17 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T17:41:17.222+0000 7fc2c838a700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:41:17.691 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:41:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:17 smithi178 ceph-mon[39811]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:17 smithi178 ceph-mon[39811]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:41:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:17 smithi178 ceph-mon[39811]: osdmap e46: 7 total, 6 up, 7 in 2024-02-14T17:41:18.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:17 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:18.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:17 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:18.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:17 smithi178 ceph-mon[30321]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:18.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:17 smithi178 ceph-mon[30321]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:41:18.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:17 smithi178 ceph-mon[30321]: osdmap e46: 7 total, 6 up, 7 in 2024-02-14T17:41:18.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:17 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:18.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:17 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:18.148 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:17 smithi184 ceph-mon[33536]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:41:18.148 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:17 smithi184 ceph-mon[33536]: from='osd.6 v1:172.21.15.184:6808/2629842997' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:41:18.148 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:17 smithi184 ceph-mon[33536]: osdmap e46: 7 total, 6 up, 7 in 2024-02-14T17:41:18.148 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:17 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:18.148 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:17 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:19.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:19 smithi184 ceph-mon[33536]: purged_snaps scrub starts 2024-02-14T17:41:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:19 smithi184 ceph-mon[33536]: purged_snaps scrub ok 2024-02-14T17:41:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:19 smithi184 ceph-mon[33536]: osd.6 v1:172.21.15.184:6808/2629842997 boot 2024-02-14T17:41:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:19 smithi184 ceph-mon[33536]: osdmap e47: 7 total, 7 up, 7 in 2024-02-14T17:41:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:19 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:19 smithi184 ceph-mon[33536]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[39811]: purged_snaps scrub starts 2024-02-14T17:41:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[39811]: purged_snaps scrub ok 2024-02-14T17:41:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[39811]: osd.6 v1:172.21.15.184:6808/2629842997 boot 2024-02-14T17:41:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[39811]: osdmap e47: 7 total, 7 up, 7 in 2024-02-14T17:41:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[39811]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[30321]: purged_snaps scrub starts 2024-02-14T17:41:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[30321]: purged_snaps scrub ok 2024-02-14T17:41:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[30321]: osd.6 v1:172.21.15.184:6808/2629842997 boot 2024-02-14T17:41:19.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[30321]: osdmap e47: 7 total, 7 up, 7 in 2024-02-14T17:41:19.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:41:19.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:19 smithi178 ceph-mon[30321]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:20.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:20 smithi184 ceph-mon[33536]: osdmap e48: 7 total, 7 up, 7 in 2024-02-14T17:41:20.421 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-14T17:41:20.422 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-14T17:41:20.422 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-02-14T17:41:20.422 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-02-14T17:41:20.422 INFO:teuthology.orchestra.run.smithi184.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0161678 s, 649 MB/s 2024-02-14T17:41:20.422 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-02-14T17:41:20.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:20 smithi178 ceph-mon[39811]: osdmap e48: 7 total, 7 up, 7 in 2024-02-14T17:41:20.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:20 smithi178 ceph-mon[30321]: osdmap e48: 7 total, 7 up, 7 in 2024-02-14T17:41:21.062 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_1 2024-02-14T17:41:21.397 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:21 smithi184 ceph-mon[33536]: osdmap e49: 7 total, 7 up, 7 in 2024-02-14T17:41:21.397 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:21 smithi184 ceph-mon[33536]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:21.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:21 smithi178 ceph-mon[39811]: osdmap e49: 7 total, 7 up, 7 in 2024-02-14T17:41:21.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:21 smithi178 ceph-mon[39811]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:21 smithi178 ceph-mon[30321]: osdmap e49: 7 total, 7 up, 7 in 2024-02-14T17:41:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:21 smithi178 ceph-mon[30321]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:21.733 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:41:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: osdmap e50: 7 total, 7 up, 7 in 2024-02-14T17:41:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: Detected new or changed devices on smithi184 2024-02-14T17:41:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi184 to 5696M 2024-02-14T17:41:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:22.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:22.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:22 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: osdmap e50: 7 total, 7 up, 7 in 2024-02-14T17:41:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: Detected new or changed devices on smithi184 2024-02-14T17:41:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi184 to 5696M 2024-02-14T17:41:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: osdmap e50: 7 total, 7 up, 7 in 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: Detected new or changed devices on smithi184 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi184 to 5696M 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:22.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:22.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:22 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:23.240 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.238+0000 7f9538504700 1 -- 172.21.15.184:0/3580693195 >> v1:172.21.15.184:6789/0 conn(0x7f9530109bf0 legacy=0x7f953010c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:23.240 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.238+0000 7f9538504700 1 -- 172.21.15.184:0/3580693195 shutdown_connections 2024-02-14T17:41:23.240 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.238+0000 7f9538504700 1 -- 172.21.15.184:0/3580693195 >> 172.21.15.184:0/3580693195 conn(0x7f95300fd020 msgr2=0x7f95300ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:23.240 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.238+0000 7f9538504700 1 -- 172.21.15.184:0/3580693195 shutdown_connections 2024-02-14T17:41:23.240 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.238+0000 7f9538504700 1 -- 172.21.15.184:0/3580693195 wait complete. 2024-02-14T17:41:23.240 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.238+0000 7f9538504700 1 Processor -- start 2024-02-14T17:41:23.240 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.238+0000 7f9538504700 1 -- start start 2024-02-14T17:41:23.240 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.238+0000 7f9538504700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9530197960 con 0x7f9530101a80 2024-02-14T17:41:23.240 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.238+0000 7f9538504700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95301a3150 con 0x7f9530105eb0 2024-02-14T17:41:23.240 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.238+0000 7f9538504700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95301a42f0 con 0x7f9530109bf0 2024-02-14T17:41:23.240 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f9535a9f700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f9530105eb0 0x7f953019d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:57880/0 (socket says 172.21.15.184:57880) 2024-02-14T17:41:23.241 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f9535a9f700 1 -- 172.21.15.184:0/2148129363 learned_addr learned my addr 172.21.15.184:0/2148129363 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:41:23.241 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070622465 0 0) 0x7f95301a3150 con 0x7f9530105eb0 2024-02-14T17:41:23.241 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f951c0035b0 con 0x7f9530105eb0 2024-02-14T17:41:23.241 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:23 smithi184 ceph-mon[33536]: osdmap e51: 7 total, 7 up, 7 in 2024-02-14T17:41:23.241 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:23 smithi184 ceph-mon[33536]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-14T17:41:23.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492745085 0 0) 0x7f95301a42f0 con 0x7f9530109bf0 2024-02-14T17:41:23.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95301a3150 con 0x7f9530109bf0 2024-02-14T17:41:23.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178969000 0 0) 0x7f9530197960 con 0x7f9530101a80 2024-02-14T17:41:23.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95301a42f0 con 0x7f9530101a80 2024-02-14T17:41:23.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 453047092 0 0) 0x7f951c0035b0 con 0x7f9530105eb0 2024-02-14T17:41:23.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9530197960 con 0x7f9530105eb0 2024-02-14T17:41:23.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f9520001930 con 0x7f9530105eb0 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 685740670 0 0) 0x7f95301a3150 con 0x7f9530109bf0 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f951c0035b0 con 0x7f9530109bf0 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f9528004830 con 0x7f9530109bf0 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1343420999 0 0) 0x7f9530197960 con 0x7f9530105eb0 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 >> v1:172.21.15.178:6790/0 conn(0x7f9530109bf0 legacy=0x7f95301a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 >> v1:172.21.15.178:6789/0 conn(0x7f9530101a80 legacy=0x7f9530196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95301a5490 con 0x7f9530105eb0 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f9538504700 1 -- 172.21.15.184:0/2148129363 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95301a44e0 con 0x7f9530105eb0 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.239+0000 7f9538504700 1 -- 172.21.15.184:0/2148129363 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f95301a49e0 con 0x7f9530105eb0 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.240+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f9520001ba0 con 0x7f9530105eb0 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.240+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f9520005ce0 con 0x7f9530105eb0 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.241+0000 7f9538504700 1 -- 172.21.15.184:0/2148129363 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9514005150 con 0x7f9530105eb0 2024-02-14T17:41:23.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.241+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7f9520003950 con 0x7f9530105eb0 2024-02-14T17:41:23.245 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.242+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 3789+0+0 (unknown 3873944933 0 0) 0x7f9520087d60 con 0x7f9530105eb0 2024-02-14T17:41:23.247 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.246+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f9520051f50 con 0x7f9530105eb0 2024-02-14T17:41:23.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:23.444+0000 7f9538504700 1 -- 172.21.15.184:0/2148129363 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9514000bc0 con 0x7f951c06e7f0 2024-02-14T17:41:23.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:23 smithi178 ceph-mon[39811]: osdmap e51: 7 total, 7 up, 7 in 2024-02-14T17:41:23.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:23 smithi178 ceph-mon[39811]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-14T17:41:23.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:23 smithi178 ceph-mon[30321]: osdmap e51: 7 total, 7 up, 7 in 2024-02-14T17:41:23.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:23 smithi178 ceph-mon[30321]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-14T17:41:24.310 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:24 smithi184 ceph-mon[33536]: from='client.24316 v1:172.21.15.184:0/2148129363' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:24.310 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:24 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:41:24.310 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:24 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:41:24.310 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:24 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:24.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:24 smithi178 ceph-mon[39811]: from='client.24316 v1:172.21.15.184:0/2148129363' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:24.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:24 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:41:24.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:24 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:41:24.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:24 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:24.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:24 smithi178 ceph-mon[30321]: from='client.24316 v1:172.21.15.184:0/2148129363' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:24.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:24 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:41:24.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:24 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:41:24.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:24 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:25.284 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:25 smithi184 ceph-mon[33536]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-14T17:41:25.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:25 smithi178 ceph-mon[39811]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-14T17:41:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:25 smithi178 ceph-mon[30321]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-02-14T17:41:26.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/1727922162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4709b8f-a0f5-4623-8b94-db6c8b59f796"}]: dispatch 2024-02-14T17:41:26.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:26 smithi184 ceph-mon[33536]: from='client.24322 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4709b8f-a0f5-4623-8b94-db6c8b59f796"}]: dispatch 2024-02-14T17:41:26.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:26 smithi184 ceph-mon[33536]: from='client.24322 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4709b8f-a0f5-4623-8b94-db6c8b59f796"}]': finished 2024-02-14T17:41:26.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:26 smithi184 ceph-mon[33536]: osdmap e52: 8 total, 7 up, 8 in 2024-02-14T17:41:26.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:26 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:26.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/1727922162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4709b8f-a0f5-4623-8b94-db6c8b59f796"}]: dispatch 2024-02-14T17:41:26.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:26 smithi178 ceph-mon[39811]: from='client.24322 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4709b8f-a0f5-4623-8b94-db6c8b59f796"}]: dispatch 2024-02-14T17:41:26.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:26 smithi178 ceph-mon[39811]: from='client.24322 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4709b8f-a0f5-4623-8b94-db6c8b59f796"}]': finished 2024-02-14T17:41:26.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:26 smithi178 ceph-mon[39811]: osdmap e52: 8 total, 7 up, 8 in 2024-02-14T17:41:26.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:26 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:26.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/1727922162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4709b8f-a0f5-4623-8b94-db6c8b59f796"}]: dispatch 2024-02-14T17:41:26.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:26 smithi178 ceph-mon[30321]: from='client.24322 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4709b8f-a0f5-4623-8b94-db6c8b59f796"}]: dispatch 2024-02-14T17:41:26.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:26 smithi178 ceph-mon[30321]: from='client.24322 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4709b8f-a0f5-4623-8b94-db6c8b59f796"}]': finished 2024-02-14T17:41:26.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:26 smithi178 ceph-mon[30321]: osdmap e52: 8 total, 7 up, 8 in 2024-02-14T17:41:26.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:26 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/2912011593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:27.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:27 smithi184 ceph-mon[33536]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T17:41:27.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/2912011593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:27.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:27 smithi178 ceph-mon[39811]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T17:41:27.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/2912011593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:27.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:27 smithi178 ceph-mon[30321]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T17:41:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:29 smithi178 ceph-mon[30321]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:30.118 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:29 smithi178 ceph-mon[39811]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:30.199 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:29 smithi184 ceph-mon[33536]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:32.161 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:31 smithi184 ceph-mon[33536]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:32.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:31 smithi178 ceph-mon[30321]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:32.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:31 smithi178 ceph-mon[39811]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:33.164 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:32 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-14T17:41:33.164 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:32 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:32 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-14T17:41:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:32 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:32 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-14T17:41:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:32 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:34.060 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:33 smithi184 ceph-mon[33536]: Deploying daemon osd.7 on smithi184 2024-02-14T17:41:34.060 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:33 smithi184 ceph-mon[33536]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:34.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:33 smithi178 ceph-mon[30321]: Deploying daemon osd.7 on smithi184 2024-02-14T17:41:34.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:33 smithi178 ceph-mon[30321]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:33 smithi178 ceph-mon[39811]: Deploying daemon osd.7 on smithi184 2024-02-14T17:41:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:33 smithi178 ceph-mon[39811]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:35.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:35 smithi184 ceph-mon[33536]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:36.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:35 smithi178 ceph-mon[30321]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:36.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:35 smithi178 ceph-mon[39811]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:38.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:37 smithi184 ceph-mon[33536]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:38.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:37 smithi178 ceph-mon[30321]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:37 smithi178 ceph-mon[39811]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:39 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:39 smithi178 ceph-mon[30321]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:39 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:39 smithi178 ceph-mon[39811]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:39.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:39 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:39.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:39 smithi184 ceph-mon[33536]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:41.558 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 7 on host 'smithi184' 2024-02-14T17:41:41.558 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:41.555+0000 7f95277fe700 1 -- 172.21.15.184:0/2148129363 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3718230236) 0x7f9514000bc0 con 0x7f951c06e7f0 2024-02-14T17:41:41.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:41.557+0000 7f9538504700 1 -- 172.21.15.184:0/2148129363 >> v1:172.21.15.178:6800/1500761607 conn(0x7f951c06e7f0 legacy=0x7f951c070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:41.560 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:41.557+0000 7f9538504700 1 -- 172.21.15.184:0/2148129363 >> v1:172.21.15.184:6789/0 conn(0x7f9530105eb0 legacy=0x7f953019d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:41.560 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:41.557+0000 7f9538504700 1 -- 172.21.15.184:0/2148129363 shutdown_connections 2024-02-14T17:41:41.560 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:41.557+0000 7f9538504700 1 -- 172.21.15.184:0/2148129363 >> 172.21.15.184:0/2148129363 conn(0x7f95300fd020 msgr2=0x7f953010c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:41.560 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:41.557+0000 7f9538504700 1 -- 172.21.15.184:0/2148129363 shutdown_connections 2024-02-14T17:41:41.560 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:41.557+0000 7f9538504700 1 -- 172.21.15.184:0/2148129363 wait complete. 2024-02-14T17:41:42.134 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:41 smithi184 ceph-mon[33536]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:42.134 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:41 smithi184 ceph-mon[33536]: from='osd.7 v1:172.21.15.184:6812/1589171201' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:41:42.134 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:41 smithi184 ceph-mon[33536]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:41:42.134 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:41 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:42.134 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:41 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:42.161 DEBUG:teuthology.orchestra.run.smithi184:osd.7> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.7.service 2024-02-14T17:41:42.164 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-14T17:41:42.164 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd stat -f json 2024-02-14T17:41:42.193 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:41 smithi178 ceph-mon[39811]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:42.194 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:41 smithi178 ceph-mon[39811]: from='osd.7 v1:172.21.15.184:6812/1589171201' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:41:42.194 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:41 smithi178 ceph-mon[39811]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:41:42.194 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:41 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:42.194 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:41 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:42.194 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:41 smithi178 ceph-mon[30321]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:42.194 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:41 smithi178 ceph-mon[30321]: from='osd.7 v1:172.21.15.184:6812/1589171201' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:41:42.194 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:41 smithi178 ceph-mon[30321]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:41:42.194 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:41 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:42.194 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:41 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:42.416 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:41:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[30321]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-14T17:41:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[30321]: osdmap e53: 8 total, 7 up, 8 in 2024-02-14T17:41:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[30321]: from='osd.7 v1:172.21.15.184:6812/1589171201' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:41:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[30321]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:41:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:43.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[39811]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-14T17:41:43.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[39811]: osdmap e53: 8 total, 7 up, 8 in 2024-02-14T17:41:43.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[39811]: from='osd.7 v1:172.21.15.184:6812/1589171201' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:41:43.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[39811]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:41:43.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:43.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:42 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:43.196 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:42 smithi184 ceph-mon[33536]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-14T17:41:43.196 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:42 smithi184 ceph-mon[33536]: osdmap e53: 8 total, 7 up, 8 in 2024-02-14T17:41:43.196 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:42 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.196 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:42 smithi184 ceph-mon[33536]: from='osd.7 v1:172.21.15.184:6812/1589171201' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:41:43.196 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:42 smithi184 ceph-mon[33536]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-14T17:41:43.197 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:42 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:43.197 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:42 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:43.470 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 17:41:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T17:41:43.281+0000 7fd84258b700 -1 osd.7 0 waiting for initial osdmap 2024-02-14T17:41:43.470 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 17:41:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T17:41:43.289+0000 7fd83a376700 -1 osd.7 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:41:43.941 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:43 smithi184 ceph-mon[33536]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:43.941 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:43 smithi184 ceph-mon[33536]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:41:43.941 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:43 smithi184 ceph-mon[33536]: osdmap e54: 8 total, 7 up, 8 in 2024-02-14T17:41:43.941 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.941 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.942 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:43 smithi184 ceph-mon[33536]: osdmap e55: 8 total, 7 up, 8 in 2024-02-14T17:41:43.942 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:43 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.942 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:43 smithi184 ceph-mon[33536]: from='osd.7 ' entity='osd.7' 2024-02-14T17:41:43.953 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[30321]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:43.953 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[30321]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:41:43.953 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[30321]: osdmap e54: 8 total, 7 up, 8 in 2024-02-14T17:41:43.953 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.953 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.953 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[30321]: osdmap e55: 8 total, 7 up, 8 in 2024-02-14T17:41:43.953 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.953 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[30321]: from='osd.7 ' entity='osd.7' 2024-02-14T17:41:43.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[39811]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:43.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[39811]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-14T17:41:43.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[39811]: osdmap e54: 8 total, 7 up, 8 in 2024-02-14T17:41:43.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[39811]: osdmap e55: 8 total, 7 up, 8 in 2024-02-14T17:41:43.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:43.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:43 smithi178 ceph-mon[39811]: from='osd.7 ' entity='osd.7' 2024-02-14T17:41:44.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.207+0000 7eff41772700 1 -- 172.21.15.178:0/1596739882 >> v1:172.21.15.178:6789/0 conn(0x7eff3c108af0 legacy=0x7eff3c10afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:44.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.207+0000 7eff41772700 1 -- 172.21.15.178:0/1596739882 shutdown_connections 2024-02-14T17:41:44.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.207+0000 7eff41772700 1 -- 172.21.15.178:0/1596739882 >> 172.21.15.178:0/1596739882 conn(0x7eff3c0fbf60 msgr2=0x7eff3c0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:44.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.207+0000 7eff41772700 1 -- 172.21.15.178:0/1596739882 shutdown_connections 2024-02-14T17:41:44.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.207+0000 7eff41772700 1 -- 172.21.15.178:0/1596739882 wait complete. 2024-02-14T17:41:44.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.208+0000 7eff41772700 1 Processor -- start 2024-02-14T17:41:44.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.208+0000 7eff41772700 1 -- start start 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.208+0000 7eff41772700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7eff3c197620 con 0x7eff3c104db0 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.208+0000 7eff41772700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7eff3c1a2e10 con 0x7eff3c100980 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.208+0000 7eff41772700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7eff3c1a3fb0 con 0x7eff3c108af0 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.208+0000 7eff3b7fe700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7eff3c108af0 0x7eff3c1a06e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:49056/0 (socket says 172.21.15.178:49056) 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.208+0000 7eff3a7fc700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7eff3c104db0 0x7eff3c19cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:49266/0 (socket says 172.21.15.178:49266) 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.208+0000 7eff3affd700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7eff3c100980 0x7eff3c196ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.178:38062/0 (socket says 172.21.15.178:38062) 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.208+0000 7eff3b7fe700 1 -- 172.21.15.178:0/3197729205 learned_addr learned my addr 172.21.15.178:0/3197729205 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.208+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928416144 0 0) 0x7eff3c1a2e10 con 0x7eff3c100980 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff24005070 con 0x7eff3c100980 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585944479 0 0) 0x7eff3c1a3fb0 con 0x7eff3c108af0 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff3c1a2e10 con 0x7eff3c108af0 2024-02-14T17:41:44.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346376491 0 0) 0x7eff3c197620 con 0x7eff3c104db0 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff3c1a3fb0 con 0x7eff3c104db0 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 999737226 0 0) 0x7eff24005070 con 0x7eff3c100980 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7eff3c197620 con 0x7eff3c100980 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7eff2c0018d0 con 0x7eff3c100980 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3482226569 0 0) 0x7eff3c1a3fb0 con 0x7eff3c104db0 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7eff24005070 con 0x7eff3c104db0 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 107708920 0 0) 0x7eff3c1a2e10 con 0x7eff3c108af0 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7eff3c1a3fb0 con 0x7eff3c108af0 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7eff300017b0 con 0x7eff3c104db0 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7eff340017b0 con 0x7eff3c108af0 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1172299251 0 0) 0x7eff3c197620 con 0x7eff3c100980 2024-02-14T17:41:44.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 >> v1:172.21.15.178:6790/0 conn(0x7eff3c108af0 legacy=0x7eff3c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:44.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 >> v1:172.21.15.178:6789/0 conn(0x7eff3c104db0 legacy=0x7eff3c19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:44.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c1a5150 con 0x7eff3c100980 2024-02-14T17:41:44.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff41772700 1 -- 172.21.15.178:0/3197729205 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c1a41a0 con 0x7eff3c100980 2024-02-14T17:41:44.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff41772700 1 -- 172.21.15.178:0/3197729205 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c1a46a0 con 0x7eff3c100980 2024-02-14T17:41:44.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7eff2c003b10 con 0x7eff3c100980 2024-02-14T17:41:44.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.209+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7eff2c005bf0 con 0x7eff3c100980 2024-02-14T17:41:44.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.210+0000 7eff41772700 1 -- 172.21.15.178:0/3197729205 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff28005150 con 0x7eff3c100980 2024-02-14T17:41:44.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.210+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7eff2c01c310 con 0x7eff3c100980 2024-02-14T17:41:44.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.211+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(56..56 src has 1..56) v4 ==== 4114+0+0 (unknown 1066518956 0 0) 0x7eff2c087ff0 con 0x7eff3c100980 2024-02-14T17:41:44.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.216+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7eff2c051d60 con 0x7eff3c100980 2024-02-14T17:41:44.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.413+0000 7eff41772700 1 -- 172.21.15.178:0/3197729205 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7eff28005400 con 0x7eff3c100980 2024-02-14T17:41:44.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.413+0000 7eff23fff700 1 -- 172.21.15.178:0/3197729205 <== mon.1 v1:172.21.15.184:6789/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v56) v1 ==== 74+0+130 (unknown 401554801 0 3400960511) 0x7eff2c053e60 con 0x7eff3c100980 2024-02-14T17:41:44.415 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:41:44.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.415+0000 7eff41772700 1 -- 172.21.15.178:0/3197729205 >> v1:172.21.15.178:6800/1500761607 conn(0x7eff2406eb40 legacy=0x7eff24071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:44.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.416+0000 7eff41772700 1 -- 172.21.15.178:0/3197729205 >> v1:172.21.15.184:6789/0 conn(0x7eff3c100980 legacy=0x7eff3c196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:44.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.416+0000 7eff41772700 1 -- 172.21.15.178:0/3197729205 shutdown_connections 2024-02-14T17:41:44.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.416+0000 7eff41772700 1 -- 172.21.15.178:0/3197729205 >> 172.21.15.178:0/3197729205 conn(0x7eff3c0fbf60 msgr2=0x7eff3c1080d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:44.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.416+0000 7eff41772700 1 -- 172.21.15.178:0/3197729205 shutdown_connections 2024-02-14T17:41:44.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:44.416+0000 7eff41772700 1 -- 172.21.15.178:0/3197729205 wait complete. 2024-02-14T17:41:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[39811]: purged_snaps scrub starts 2024-02-14T17:41:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[39811]: purged_snaps scrub ok 2024-02-14T17:41:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[39811]: osd.7 v1:172.21.15.184:6812/1589171201 boot 2024-02-14T17:41:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[39811]: osdmap e56: 8 total, 8 up, 8 in 2024-02-14T17:41:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3197729205' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:41:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[30321]: purged_snaps scrub starts 2024-02-14T17:41:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[30321]: purged_snaps scrub ok 2024-02-14T17:41:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[30321]: osd.7 v1:172.21.15.184:6812/1589171201 boot 2024-02-14T17:41:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[30321]: osdmap e56: 8 total, 8 up, 8 in 2024-02-14T17:41:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3197729205' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:41:45.155 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1707932504,"num_in_osds":8,"osd_in_since":1707932486,"num_remapped_pgs":1} 2024-02-14T17:41:45.156 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T17:41:45.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:44 smithi184 ceph-mon[33536]: purged_snaps scrub starts 2024-02-14T17:41:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:44 smithi184 ceph-mon[33536]: purged_snaps scrub ok 2024-02-14T17:41:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:44 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:44 smithi184 ceph-mon[33536]: osd.7 v1:172.21.15.184:6812/1589171201 boot 2024-02-14T17:41:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:44 smithi184 ceph-mon[33536]: osdmap e56: 8 total, 8 up, 8 in 2024-02-14T17:41:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:44 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:41:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3197729205' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:41:45.436 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:41:45.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: osdmap e57: 8 total, 8 up, 8 in 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:45.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:45.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:45 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: osdmap e57: 8 total, 8 up, 8 in 2024-02-14T17:41:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:46.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:46.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:46.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:46.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:46.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:46.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:41:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: osdmap e57: 8 total, 8 up, 8 in 2024-02-14T17:41:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:46.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:46.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:46.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:46.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:46.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:46.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:46.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:46.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:46.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:46.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:46.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:46.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:45 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:47.095 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:46 smithi178 ceph-mon[39811]: Detected new or changed devices on smithi184 2024-02-14T17:41:47.095 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:46 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi184 to 4272M 2024-02-14T17:41:47.095 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:46 smithi178 ceph-mon[39811]: osdmap e58: 8 total, 8 up, 8 in 2024-02-14T17:41:47.095 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:46 smithi178 ceph-mon[30321]: Detected new or changed devices on smithi184 2024-02-14T17:41:47.096 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:46 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi184 to 4272M 2024-02-14T17:41:47.096 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:46 smithi178 ceph-mon[30321]: osdmap e58: 8 total, 8 up, 8 in 2024-02-14T17:41:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.090+0000 7fb52433a700 1 -- 172.21.15.178:0/609058882 >> v1:172.21.15.178:6789/0 conn(0x7fb51c10e4b0 legacy=0x7fb51c110970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.090+0000 7fb52433a700 1 -- 172.21.15.178:0/609058882 shutdown_connections 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.090+0000 7fb52433a700 1 -- 172.21.15.178:0/609058882 >> 172.21.15.178:0/609058882 conn(0x7fb51c0fd560 msgr2=0x7fb51c0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.090+0000 7fb52433a700 1 -- 172.21.15.178:0/609058882 shutdown_connections 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.090+0000 7fb52433a700 1 -- 172.21.15.178:0/609058882 wait complete. 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb52433a700 1 Processor -- start 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb52433a700 1 -- start start 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb52433a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb51c1a8b60 con 0x7fb51c10e4b0 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb52433a700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb51c1a9d40 con 0x7fb51c10a8e0 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb52433a700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb51c1aaf20 con 0x7fb51c107a30 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb5220d6700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7fb51c107a30 0x7fb51c0758c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:49080/0 (socket says 172.21.15.178:49080) 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb5228d7700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fb51c10e4b0 0x7fb51c1a7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:49296/0 (socket says 172.21.15.178:49296) 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb5220d6700 1 -- 172.21.15.178:0/506215146 learned_addr learned my addr 172.21.15.178:0/506215146 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837158607 0 0) 0x7fb51c1a8b60 con 0x7fb51c10e4b0 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5080035b0 con 0x7fb51c10e4b0 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526233487 0 0) 0x7fb51c1aaf20 con 0x7fb51c107a30 2024-02-14T17:41:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb51c1a8b60 con 0x7fb51c107a30 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.091+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032970650 0 0) 0x7fb51c1a9d40 con 0x7fb51c10a8e0 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb51c1aaf20 con 0x7fb51c10a8e0 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1633151765 0 0) 0x7fb51c1a8b60 con 0x7fb51c107a30 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb51c1a9d40 con 0x7fb51c107a30 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3094119297 0 0) 0x7fb5080035b0 con 0x7fb51c10e4b0 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb51c1a8b60 con 0x7fb51c10e4b0 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fb50c001870 con 0x7fb51c107a30 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fb514004800 con 0x7fb51c10e4b0 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3190901431 0 0) 0x7fb51c1aaf20 con 0x7fb51c10a8e0 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5080035b0 con 0x7fb51c10a8e0 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fb518000ef0 con 0x7fb51c10a8e0 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2230083243 0 0) 0x7fb51c1a8b60 con 0x7fb51c10e4b0 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 >> v1:172.21.15.178:6790/0 conn(0x7fb51c107a30 legacy=0x7fb51c0758c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 >> v1:172.21.15.184:6789/0 conn(0x7fb51c10a8e0 legacy=0x7fb51c1a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c1ac120 con 0x7fb51c10e4b0 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb52433a700 1 -- 172.21.15.178:0/506215146 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c1ab110 con 0x7fb51c10e4b0 2024-02-14T17:41:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb52433a700 1 -- 172.21.15.178:0/506215146 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c1ab590 con 0x7fb51c10e4b0 2024-02-14T17:41:47.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fb514001520 con 0x7fb51c10e4b0 2024-02-14T17:41:47.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.092+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fb514004fe0 con 0x7fb51c10e4b0 2024-02-14T17:41:47.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.093+0000 7fb52433a700 1 -- 172.21.15.178:0/506215146 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb500005150 con 0x7fb51c10e4b0 2024-02-14T17:41:47.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.094+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7fb514005140 con 0x7fb51c10e4b0 2024-02-14T17:41:47.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.094+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(58..58 src has 1..58) v4 ==== 4110+0+0 (unknown 1639750134 0 0) 0x7fb514088260 con 0x7fb51c10e4b0 2024-02-14T17:41:47.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.101+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fb514052210 con 0x7fb51c10e4b0 2024-02-14T17:41:47.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:46 smithi184 ceph-mon[33536]: Detected new or changed devices on smithi184 2024-02-14T17:41:47.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:46 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi184 to 4272M 2024-02-14T17:41:47.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:46 smithi184 ceph-mon[33536]: osdmap e58: 8 total, 8 up, 8 in 2024-02-14T17:41:47.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.295+0000 7fb52433a700 1 -- 172.21.15.178:0/506215146 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb500005400 con 0x7fb51c10e4b0 2024-02-14T17:41:47.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.296+0000 7fb5137fe700 1 -- 172.21.15.178:0/506215146 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+11535 (unknown 1651271748 0 3334764259) 0x7fb514054140 con 0x7fb51c10e4b0 2024-02-14T17:41:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:41:47.298 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":59,"fsid":"6e1588ec-cb5f-11ee-95b9-87774f69a715","created":"2024-02-14T17:36:20.333938+0000","modified":"2024-02-14T17:41:47.160356+0000","last_up_change":"2024-02-14T17:41:44.153032+0000","last_in_change":"2024-02-14T17:41:26.094057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T17:39:39.062242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96b590dc-522b-4b8d-938b-5d3de2944e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6801","nonce":2284738682}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6802","nonce":2284738682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6804","nonce":2284738682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6803","nonce":2284738682}]},"public_addr":"172.21.15.178:6801/2284738682","cluster_addr":"172.21.15.178:6802/2284738682","heartbeat_back_addr":"172.21.15.178:6804/2284738682","heartbeat_front_addr":"172.21.15.178:6803/2284738682","state":["exists","up"]},{"osd":1,"uuid":"c97cb851-c6dd-4230-ace7-1fe481f29ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6805","nonce":3678228960}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6806","nonce":3678228960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6808","nonce":3678228960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6807","nonce":3678228960}]},"public_addr":"172.21.15.178:6805/3678228960","cluster_addr":"172.21.15.178:6806/3678228960","heartbeat_back_addr":"172.21.15.178:6808/3678228960","heartbeat_front_addr":"172.21.15.178:6807/3678228960","state":["exists","up"]},{"osd":2,"uuid":"bebc9b6f-6349-4f8e-83b1-4177c816c21f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6809","nonce":479756253}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6810","nonce":479756253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6812","nonce":479756253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6811","nonce":479756253}]},"public_addr":"172.21.15.178:6809/479756253","cluster_addr":"172.21.15.178:6810/479756253","heartbeat_back_addr":"172.21.15.178:6812/479756253","heartbeat_front_addr":"172.21.15.178:6811/479756253","state":["exists","up"]},{"osd":3,"uuid":"058ca4e0-7fc0-4b14-93ef-fd9fa2fecd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6813","nonce":731412783}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6814","nonce":731412783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6816","nonce":731412783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6815","nonce":731412783}]},"public_addr":"172.21.15.178:6813/731412783","cluster_addr":"172.21.15.178:6814/731412783","heartbeat_back_addr":"172.21.15.178:6816/731412783","heartbeat_front_addr":"172.21.15.178:6815/731412783","state":["exists","up"]},{"osd":4,"uuid":"c0ea5121-5d8f-48ae-84cd-ba34c8efa7af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6800","nonce":3570504556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6801","nonce":3570504556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6803","nonce":3570504556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6802","nonce":3570504556}]},"public_addr":"172.21.15.184:6800/3570504556","cluster_addr":"172.21.15.184:6801/3570504556","heartbeat_back_addr":"172.21.15.184:6803/3570504556","heartbeat_front_addr":"172.21.15.184:6802/3570504556","state":["exists","up"]},{"osd":5,"uuid":"165cded3-2161-4e1c-93f4-719dd6c7cc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6804","nonce":2863982197}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6805","nonce":2863982197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6807","nonce":2863982197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6806","nonce":2863982197}]},"public_addr":"172.21.15.184:6804/2863982197","cluster_addr":"172.21.15.184:6805/2863982197","heartbeat_back_addr":"172.21.15.184:6807/2863982197","heartbeat_front_addr":"172.21.15.184:6806/2863982197","state":["exists","up"]},{"osd":6,"uuid":"93c78324-6494-44c1-96b1-4c957ae7b33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6808","nonce":2629842997}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6809","nonce":2629842997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6811","nonce":2629842997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6810","nonce":2629842997}]},"public_addr":"172.21.15.184:6808/2629842997","cluster_addr":"172.21.15.184:6809/2629842997","heartbeat_back_addr":"172.21.15.184:6811/2629842997","heartbeat_front_addr":"172.21.15.184:6810/2629842997","state":["exists","up"]},{"osd":7,"uuid":"f4709b8f-a0f5-4623-8b94-db6c8b59f796","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6812","nonce":1589171201}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6813","nonce":1589171201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6815","nonce":1589171201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6814","nonce":1589171201}]},"public_addr":"172.21.15.184:6812/1589171201","cluster_addr":"172.21.15.184:6813/1589171201","heartbeat_back_addr":"172.21.15.184:6815/1589171201","heartbeat_front_addr":"172.21.15.184:6814/1589171201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:38:43.720721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:39:08.896963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:39:34.807815+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:40:01.129025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:40:25.125752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:40:50.590836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:41:16.451927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:41:42.151194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:0/2537478068":"2024-02-15T17:37:20.729807+0000","172.21.15.178:0/373398875":"2024-02-15T17:36:53.862750+0000","172.21.15.178:0/1473396399":"2024-02-15T17:37:20.729807+0000","172.21.15.178:0/2817516403":"2024-02-15T17:36:53.862750+0000","172.21.15.178:6800/1774918016":"2024-02-15T17:36:53.862750+0000","172.21.15.178:0/3369364101":"2024-02-15T17:37:20.729807+0000","172.21.15.178:0/1879459737":"2024-02-15T17:36:53.862750+0000","172.21.15.178:6800/366374299":"2024-02-15T17:37:20.729807+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T17:41:47.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.298+0000 7fb52433a700 1 -- 172.21.15.178:0/506215146 >> v1:172.21.15.178:6800/1500761607 conn(0x7fb50806ea10 legacy=0x7fb508070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:47.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.298+0000 7fb52433a700 1 -- 172.21.15.178:0/506215146 >> v1:172.21.15.178:6789/0 conn(0x7fb51c10e4b0 legacy=0x7fb51c1a7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:47.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.298+0000 7fb52433a700 1 -- 172.21.15.178:0/506215146 shutdown_connections 2024-02-14T17:41:47.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.298+0000 7fb52433a700 1 -- 172.21.15.178:0/506215146 >> 172.21.15.178:0/506215146 conn(0x7fb51c0fd560 msgr2=0x7fb51c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:47.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.299+0000 7fb52433a700 1 -- 172.21.15.178:0/506215146 shutdown_connections 2024-02-14T17:41:47.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:47.299+0000 7fb52433a700 1 -- 172.21.15.178:0/506215146 wait complete. 2024-02-14T17:41:47.998 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T17:39:39.062242+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-14T17:41:47.998 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-14T17:41:48.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:47 smithi184 ceph-mon[33536]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:41:48.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:47 smithi184 ceph-mon[33536]: osdmap e59: 8 total, 8 up, 8 in 2024-02-14T17:41:48.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/506215146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:41:48.239 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:41:48.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:47 smithi178 ceph-mon[30321]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:41:48.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:47 smithi178 ceph-mon[30321]: osdmap e59: 8 total, 8 up, 8 in 2024-02-14T17:41:48.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/506215146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:41:48.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:47 smithi178 ceph-mon[39811]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:41:48.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:47 smithi178 ceph-mon[39811]: osdmap e59: 8 total, 8 up, 8 in 2024-02-14T17:41:48.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/506215146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:41:49.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:49 smithi184 ceph-mon[33536]: osdmap e60: 8 total, 8 up, 8 in 2024-02-14T17:41:49.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:49 smithi184 ceph-mon[33536]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:41:49.537 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:49 smithi178 ceph-mon[39811]: osdmap e60: 8 total, 8 up, 8 in 2024-02-14T17:41:49.537 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:49 smithi178 ceph-mon[39811]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:41:49.537 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:49 smithi178 ceph-mon[30321]: osdmap e60: 8 total, 8 up, 8 in 2024-02-14T17:41:49.537 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:49 smithi178 ceph-mon[30321]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:41:50.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.058+0000 7f87ae904700 1 -- 172.21.15.178:0/1172811545 >> v1:172.21.15.178:6789/0 conn(0x7f87a810d450 legacy=0x7f87a810f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:50.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.058+0000 7f87ae904700 1 -- 172.21.15.178:0/1172811545 shutdown_connections 2024-02-14T17:41:50.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.058+0000 7f87ae904700 1 -- 172.21.15.178:0/1172811545 >> 172.21.15.178:0/1172811545 conn(0x7f87a8076da0 msgr2=0x7f87a80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:50.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.058+0000 7f87ae904700 1 -- 172.21.15.178:0/1172811545 shutdown_connections 2024-02-14T17:41:50.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.058+0000 7f87ae904700 1 -- 172.21.15.178:0/1172811545 wait complete. 2024-02-14T17:41:50.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.059+0000 7f87ae904700 1 Processor -- start 2024-02-14T17:41:50.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.059+0000 7f87ae904700 1 -- start start 2024-02-14T17:41:50.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.059+0000 7f87ae904700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f87a81a6960 con 0x7f87a8109750 2024-02-14T17:41:50.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.059+0000 7f87ae904700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f87a81a7b40 con 0x7f87a81058f0 2024-02-14T17:41:50.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.059+0000 7f87ae904700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f87a81a8d40 con 0x7f87a810d450 2024-02-14T17:41:50.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.059+0000 7f87a77fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f87a8109750 0x7f87a81a17f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:49308/0 (socket says 172.21.15.178:49308) 2024-02-14T17:41:50.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.059+0000 7f87acea1700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f87a810d450 0x7f87a81a5020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:49090/0 (socket says 172.21.15.178:49090) 2024-02-14T17:41:50.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.059+0000 7f87a77fe700 1 -- 172.21.15.178:0/1622005568 learned_addr learned my addr 172.21.15.178:0/1622005568 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:41:50.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284787689 0 0) 0x7f87a81a6960 con 0x7f87a8109750 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87900035b0 con 0x7f87a8109750 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894390527 0 0) 0x7f87a81a8d40 con 0x7f87a810d450 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a81a6960 con 0x7f87a810d450 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997898661 0 0) 0x7f87a81a7b40 con 0x7f87a81058f0 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a81a8d40 con 0x7f87a81058f0 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1941407870 0 0) 0x7f87900035b0 con 0x7f87a8109750 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87a81a7b40 con 0x7f87a8109750 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f879c001950 con 0x7f87a8109750 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4190419107 0 0) 0x7f87a81a6960 con 0x7f87a810d450 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87900035b0 con 0x7f87a810d450 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2809412069 0 0) 0x7f87a81a8d40 con 0x7f87a81058f0 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87a81a6960 con 0x7f87a81058f0 2024-02-14T17:41:50.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f87a0001890 con 0x7f87a810d450 2024-02-14T17:41:50.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f87980017f0 con 0x7f87a81058f0 2024-02-14T17:41:50.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3135819932 0 0) 0x7f87a81a7b40 con 0x7f87a8109750 2024-02-14T17:41:50.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 >> v1:172.21.15.178:6790/0 conn(0x7f87a810d450 legacy=0x7f87a81a5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:50.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 >> v1:172.21.15.184:6789/0 conn(0x7f87a81058f0 legacy=0x7f87a80fef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:50.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a81a9f40 con 0x7f87a8109750 2024-02-14T17:41:50.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87ae904700 1 -- 172.21.15.178:0/1622005568 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a81a6b50 con 0x7f87a8109750 2024-02-14T17:41:50.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87ae904700 1 -- 172.21.15.178:0/1622005568 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a81a6fd0 con 0x7f87a8109750 2024-02-14T17:41:50.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f879c003b10 con 0x7f87a8109750 2024-02-14T17:41:50.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.060+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f879c005ba0 con 0x7f87a8109750 2024-02-14T17:41:50.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.061+0000 7f87ae904700 1 -- 172.21.15.178:0/1622005568 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8794005120 con 0x7f87a8109750 2024-02-14T17:41:50.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.062+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7f879c003680 con 0x7f87a8109750 2024-02-14T17:41:50.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.062+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 4081+0+0 (unknown 1334137236 0 0) 0x7f879c087f30 con 0x7f87a8109750 2024-02-14T17:41:50.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.067+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f879c051d20 con 0x7f87a8109750 2024-02-14T17:41:50.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.257+0000 7f87ae904700 1 -- 172.21.15.178:0/1622005568 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8794005950 con 0x7f87a8109750 2024-02-14T17:41:50.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.258+0000 7f87a57fa700 1 -- 172.21.15.178:0/1622005568 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (unknown 3646463993 0 2170607528) 0x7f879c053e20 con 0x7f87a8109750 2024-02-14T17:41:50.259 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2024-02-14T17:41:50.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.260+0000 7f87ae904700 1 -- 172.21.15.178:0/1622005568 >> v1:172.21.15.178:6800/1500761607 conn(0x7f879006ea10 legacy=0x7f8790070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:50.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.260+0000 7f87ae904700 1 -- 172.21.15.178:0/1622005568 >> v1:172.21.15.178:6789/0 conn(0x7f87a8109750 legacy=0x7f87a81a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:50.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.260+0000 7f87ae904700 1 -- 172.21.15.178:0/1622005568 shutdown_connections 2024-02-14T17:41:50.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.260+0000 7f87ae904700 1 -- 172.21.15.178:0/1622005568 >> 172.21.15.178:0/1622005568 conn(0x7f87a8076da0 msgr2=0x7f87a81100e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:50.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.260+0000 7f87ae904700 1 -- 172.21.15.178:0/1622005568 shutdown_connections 2024-02-14T17:41:50.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:41:50.260+0000 7f87ae904700 1 -- 172.21.15.178:0/1622005568 wait complete. 2024-02-14T17:41:50.560 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1622005568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T17:41:50.561 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1622005568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T17:41:50.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1622005568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T17:41:50.897 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi178 2024-02-14T17:41:50.898 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi178=foo.a' 2024-02-14T17:41:51.159 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:41:51.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:51 smithi178 ceph-mon[30321]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-02-14T17:41:51.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:51 smithi178 ceph-mon[39811]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-02-14T17:41:51.593 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:51 smithi184 ceph-mon[33536]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-02-14T17:41:52.766 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.761+0000 7fee824e2700 1 -- 172.21.15.184:0/2898270601 >> v1:172.21.15.184:6789/0 conn(0x7fee7c10c0b0 legacy=0x7fee7c10e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:52.766 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.761+0000 7fee824e2700 1 -- 172.21.15.184:0/2898270601 shutdown_connections 2024-02-14T17:41:52.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.761+0000 7fee824e2700 1 -- 172.21.15.184:0/2898270601 >> 172.21.15.184:0/2898270601 conn(0x7fee7c0fbf30 msgr2=0x7fee7c0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:52.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.761+0000 7fee824e2700 1 -- 172.21.15.184:0/2898270601 shutdown_connections 2024-02-14T17:41:52.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.761+0000 7fee824e2700 1 -- 172.21.15.184:0/2898270601 wait complete. 2024-02-14T17:41:52.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.762+0000 7fee824e2700 1 Processor -- start 2024-02-14T17:41:52.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.762+0000 7fee824e2700 1 -- start start 2024-02-14T17:41:52.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.762+0000 7fee824e2700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee7c110bd0 con 0x7fee7c101e90 2024-02-14T17:41:52.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.762+0000 7fee824e2700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee7c199da0 con 0x7fee7c10c0b0 2024-02-14T17:41:52.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.762+0000 7fee824e2700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee7c1a6540 con 0x7fee7c100350 2024-02-14T17:41:52.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.762+0000 7fee80a7f700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fee7c10c0b0 0x7fee7c1a2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:33736/0 (socket says 172.21.15.184:33736) 2024-02-14T17:41:52.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.762+0000 7fee80a7f700 1 -- 172.21.15.184:0/402250406 learned_addr learned my addr 172.21.15.184:0/402250406 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:41:52.767 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79924419 0 0) 0x7fee7c199da0 con 0x7fee7c10c0b0 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee640035b0 con 0x7fee7c10c0b0 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711116840 0 0) 0x7fee7c110bd0 con 0x7fee7c101e90 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee7c199da0 con 0x7fee7c101e90 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853099693 0 0) 0x7fee7c1a6540 con 0x7fee7c100350 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee7c110bd0 con 0x7fee7c100350 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2809460907 0 0) 0x7fee640035b0 con 0x7fee7c10c0b0 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee7c1a6540 con 0x7fee7c10c0b0 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fee740017b0 con 0x7fee7c10c0b0 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3092397183 0 0) 0x7fee7c199da0 con 0x7fee7c101e90 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee640035b0 con 0x7fee7c101e90 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3087536402 0 0) 0x7fee7c110bd0 con 0x7fee7c100350 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee7c199da0 con 0x7fee7c100350 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fee700017a0 con 0x7fee7c101e90 2024-02-14T17:41:52.768 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fee6c0017b0 con 0x7fee7c100350 2024-02-14T17:41:52.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2447682607 0 0) 0x7fee7c1a6540 con 0x7fee7c10c0b0 2024-02-14T17:41:52.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 >> v1:172.21.15.178:6790/0 conn(0x7fee7c100350 legacy=0x7fee7c1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:52.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 >> v1:172.21.15.178:6789/0 conn(0x7fee7c101e90 legacy=0x7fee7c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:52.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c1a76e0 con 0x7fee7c10c0b0 2024-02-14T17:41:52.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee824e2700 1 -- 172.21.15.184:0/402250406 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7c1a6730 con 0x7fee7c10c0b0 2024-02-14T17:41:52.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee824e2700 1 -- 172.21.15.184:0/402250406 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c1a6c30 con 0x7fee7c10c0b0 2024-02-14T17:41:52.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fee74001a20 con 0x7fee7c10c0b0 2024-02-14T17:41:52.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.763+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fee74005340 con 0x7fee7c10c0b0 2024-02-14T17:41:52.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.764+0000 7fee824e2700 1 -- 172.21.15.184:0/402250406 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee68005150 con 0x7fee7c10c0b0 2024-02-14T17:41:52.769 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.765+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7fee74005540 con 0x7fee7c10c0b0 2024-02-14T17:41:52.770 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.765+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 4081+0+0 (unknown 1334137236 0 0) 0x7fee740885d0 con 0x7fee7c10c0b0 2024-02-14T17:41:52.772 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.770+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fee74052590 con 0x7fee7c10c0b0 2024-02-14T17:41:52.976 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.974+0000 7fee824e2700 1 -- 172.21.15.184:0/402250406 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi178=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fee68000bc0 con 0x7fee6406e950 2024-02-14T17:41:52.983 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.981+0000 7fee797fa700 1 -- 172.21.15.184:0/402250406 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (unknown 0 0 1123153589) 0x7fee68000bc0 con 0x7fee6406e950 2024-02-14T17:41:52.983 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled rgw.foo.a update... 2024-02-14T17:41:52.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.983+0000 7fee824e2700 1 -- 172.21.15.184:0/402250406 >> v1:172.21.15.178:6800/1500761607 conn(0x7fee6406e950 legacy=0x7fee64070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:52.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.983+0000 7fee824e2700 1 -- 172.21.15.184:0/402250406 >> v1:172.21.15.184:6789/0 conn(0x7fee7c10c0b0 legacy=0x7fee7c1a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:52.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.983+0000 7fee824e2700 1 -- 172.21.15.184:0/402250406 shutdown_connections 2024-02-14T17:41:52.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.983+0000 7fee824e2700 1 -- 172.21.15.184:0/402250406 >> 172.21.15.184:0/402250406 conn(0x7fee7c0fbf30 msgr2=0x7fee7c0fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:52.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.983+0000 7fee824e2700 1 -- 172.21.15.184:0/402250406 shutdown_connections 2024-02-14T17:41:52.986 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:52.983+0000 7fee824e2700 1 -- 172.21.15.184:0/402250406 wait complete. 2024-02-14T17:41:53.650 DEBUG:teuthology.orchestra.run.smithi178:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@rgw.foo.a.service 2024-02-14T17:41:53.652 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi184 2024-02-14T17:41:53.653 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-02-14T17:41:53.889 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:41:53.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:53 smithi184 ceph-mon[33536]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-14T17:41:53.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:53 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:53.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:53 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:53.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:53 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:53.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:53 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:53.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:53 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:53.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:53 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-14T17:41:53.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:53 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-14T17:41:53.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:53 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:53.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:53 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[30321]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-14T17:41:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-14T17:41:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-14T17:41:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:54.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:54.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[39811]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-02-14T17:41:54.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:54.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:54.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:54.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:54.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:54.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-14T17:41:54.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-14T17:41:54.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:54.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:53 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:54.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:54 smithi184 ceph-mon[33536]: from='client.24361 v1:172.21.15.184:0/402250406' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi178=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:54.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:54 smithi184 ceph-mon[33536]: Saving service rgw.foo.a spec with placement smithi178=foo.a;count:1 2024-02-14T17:41:54.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:54 smithi184 ceph-mon[33536]: Deploying daemon rgw.foo.a on smithi178 2024-02-14T17:41:55.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:54 smithi178 ceph-mon[30321]: from='client.24361 v1:172.21.15.184:0/402250406' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi178=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:55.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:54 smithi178 ceph-mon[30321]: Saving service rgw.foo.a spec with placement smithi178=foo.a;count:1 2024-02-14T17:41:55.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:54 smithi178 ceph-mon[30321]: Deploying daemon rgw.foo.a on smithi178 2024-02-14T17:41:55.305 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:54 smithi178 ceph-mon[39811]: from='client.24361 v1:172.21.15.184:0/402250406' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi178=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:55.305 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:54 smithi178 ceph-mon[39811]: Saving service rgw.foo.a spec with placement smithi178=foo.a;count:1 2024-02-14T17:41:55.305 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:54 smithi178 ceph-mon[39811]: Deploying daemon rgw.foo.a on smithi178 2024-02-14T17:41:55.525 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.519+0000 7fccee966700 1 -- 172.21.15.184:0/2916106216 >> v1:172.21.15.184:6789/0 conn(0x7fcce8104dc0 legacy=0x7fcce8107210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:55.525 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.519+0000 7fccee966700 1 -- 172.21.15.184:0/2916106216 shutdown_connections 2024-02-14T17:41:55.525 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.519+0000 7fccee966700 1 -- 172.21.15.184:0/2916106216 >> 172.21.15.184:0/2916106216 conn(0x7fcce80fbf30 msgr2=0x7fcce80fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:55.525 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.519+0000 7fccee966700 1 -- 172.21.15.184:0/2916106216 shutdown_connections 2024-02-14T17:41:55.525 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.519+0000 7fccee966700 1 -- 172.21.15.184:0/2916106216 wait complete. 2024-02-14T17:41:55.525 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.520+0000 7fccee966700 1 Processor -- start 2024-02-14T17:41:55.525 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.520+0000 7fccee966700 1 -- start start 2024-02-14T17:41:55.525 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.520+0000 7fccee966700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcce810fc10 con 0x7fcce8104dc0 2024-02-14T17:41:55.525 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.520+0000 7fccee966700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcce81a5340 con 0x7fcce8108b00 2024-02-14T17:41:55.525 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.520+0000 7fccee966700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcce81a6540 con 0x7fcce8100990 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.520+0000 7fccecf03700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fcce8108b00 0x7fcce81a29d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:33754/0 (socket says 172.21.15.184:33754) 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.520+0000 7fccecf03700 1 -- 172.21.15.184:0/3401529519 learned_addr learned my addr 172.21.15.184:0/3401529519 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.520+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326084785 0 0) 0x7fcce810fc10 con 0x7fcce8104dc0 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccd00035b0 con 0x7fcce8104dc0 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923629545 0 0) 0x7fcce81a5340 con 0x7fcce8108b00 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce810fc10 con 0x7fcce8108b00 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747667654 0 0) 0x7fcce81a6540 con 0x7fcce8100990 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce81a5340 con 0x7fcce8100990 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4173539826 0 0) 0x7fccd00035b0 con 0x7fcce8104dc0 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcce81a6540 con 0x7fcce8104dc0 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fccdc001880 con 0x7fcce8104dc0 2024-02-14T17:41:55.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2267581192 0 0) 0x7fcce810fc10 con 0x7fcce8108b00 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fccd00035b0 con 0x7fcce8108b00 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 204763865 0 0) 0x7fcce81a5340 con 0x7fcce8100990 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcce810fc10 con 0x7fcce8100990 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fcce0001c70 con 0x7fcce8108b00 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fccd8001720 con 0x7fcce8100990 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2431587791 0 0) 0x7fcce81a6540 con 0x7fcce8104dc0 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 >> v1:172.21.15.178:6790/0 conn(0x7fcce8100990 legacy=0x7fcce8106c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 >> v1:172.21.15.184:6789/0 conn(0x7fcce8108b00 legacy=0x7fcce81a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce81a7740 con 0x7fcce8104dc0 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fccee966700 1 -- 172.21.15.184:0/3401529519 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce81a4330 con 0x7fcce8104dc0 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fccee966700 1 -- 172.21.15.184:0/3401529519 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce81a4830 con 0x7fcce8104dc0 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fccdc001b40 con 0x7fcce8104dc0 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.521+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fccdc004f60 con 0x7fcce8104dc0 2024-02-14T17:41:55.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.522+0000 7fccee966700 1 -- 172.21.15.184:0/3401529519 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd4005120 con 0x7fcce8104dc0 2024-02-14T17:41:55.528 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.523+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7fccdc003860 con 0x7fcce8104dc0 2024-02-14T17:41:55.528 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.523+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 4081+0+0 (unknown 1334137236 0 0) 0x7fccdc087120 con 0x7fcce8104dc0 2024-02-14T17:41:55.530 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.528+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fccdc051080 con 0x7fcce8104dc0 2024-02-14T17:41:55.721 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:55.718+0000 7fccee966700 1 -- 172.21.15.184:0/3401529519 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7fccd4005950 con 0x7fcce8104dc0 2024-02-14T17:41:55.833 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 17:41:55 smithi178 systemd[1]: Starting Ceph rgw.foo.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:41:56.019 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:56.017+0000 7fcce57fa700 1 -- 172.21.15.184:0/3401529519 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v61) v1 ==== 160+0+0 (unknown 854540649 0 0) 0x7fccdc052fb0 con 0x7fcce8104dc0 2024-02-14T17:41:56.020 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:55 smithi184 ceph-mon[33536]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:41:56.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:56.020+0000 7fccee966700 1 -- 172.21.15.184:0/3401529519 >> v1:172.21.15.178:6800/1500761607 conn(0x7fccd006ea50 legacy=0x7fccd0070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:56.022 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:56.020+0000 7fccee966700 1 -- 172.21.15.184:0/3401529519 >> v1:172.21.15.178:6789/0 conn(0x7fcce8104dc0 legacy=0x7fcce819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:56.023 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:56.020+0000 7fccee966700 1 -- 172.21.15.184:0/3401529519 shutdown_connections 2024-02-14T17:41:56.023 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:56.020+0000 7fccee966700 1 -- 172.21.15.184:0/3401529519 >> 172.21.15.184:0/3401529519 conn(0x7fcce80fbf30 msgr2=0x7fcce806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:56.023 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:56.020+0000 7fccee966700 1 -- 172.21.15.184:0/3401529519 shutdown_connections 2024-02-14T17:41:56.023 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:41:56.020+0000 7fccee966700 1 -- 172.21.15.184:0/3401529519 wait complete. 2024-02-14T17:41:56.023 INFO:teuthology.orchestra.run.smithi184.stderr:pool 'datapool' created 2024-02-14T17:41:56.219 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:55 smithi178 ceph-mon[39811]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:41:56.220 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:55 smithi178 ceph-mon[30321]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:41:56.220 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 17:41:55 smithi178 podman[68765]: 2024-02-14 17:41:55.964760216 +0000 UTC m=+0.106721947 container create a7caf9b666504047af6a0c1298f5e995210aadce4abd2df592a3e4dbd7f4171b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-rgw-foo-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-14T17:41:56.220 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 17:41:55 smithi178 podman[68765]: 2024-02-14 17:41:55.875486964 +0000 UTC m=+0.017448703 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:41:56.220 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 17:41:56 smithi178 podman[68765]: 2024-02-14 17:41:56.219043981 +0000 UTC m=+0.361005721 container init a7caf9b666504047af6a0c1298f5e995210aadce4abd2df592a3e4dbd7f4171b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-02-14T17:41:56.582 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 17:41:56 smithi178 podman[68765]: 2024-02-14 17:41:56.224212841 +0000 UTC m=+0.366174574 container start a7caf9b666504047af6a0c1298f5e995210aadce4abd2df592a3e4dbd7f4171b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-rgw-foo-a, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:41:56.583 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 17:41:56 smithi178 bash[68765]: a7caf9b666504047af6a0c1298f5e995210aadce4abd2df592a3e4dbd7f4171b 2024-02-14T17:41:56.583 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 17:41:56 smithi178 systemd[1]: Started Ceph rgw.foo.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:41:56.626 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- rbd pool init datapool 2024-02-14T17:41:56.863 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:41:57.262 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/3401529519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-14T17:41:57.263 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/3401529519' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-14T17:41:57.263 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:56 smithi184 ceph-mon[33536]: osdmap e61: 8 total, 8 up, 8 in 2024-02-14T17:41:57.263 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.263 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.263 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.263 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:56 smithi184 ceph-mon[33536]: Saving service rgw.foo.a spec with placement smithi178=foo.a;count:1 2024-02-14T17:41:57.263 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.263 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.263 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:56 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:57.263 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:56 smithi184 ceph-mon[33536]: pgmap v189: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-14T17:41:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/3401529519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-14T17:41:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/3401529519' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-14T17:41:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[30321]: osdmap e61: 8 total, 8 up, 8 in 2024-02-14T17:41:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[30321]: Saving service rgw.foo.a spec with placement smithi178=foo.a;count:1 2024-02-14T17:41:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[30321]: pgmap v189: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-14T17:41:57.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/3401529519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-14T17:41:57.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/3401529519' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-14T17:41:57.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[39811]: osdmap e61: 8 total, 8 up, 8 in 2024-02-14T17:41:57.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[39811]: Saving service rgw.foo.a spec with placement smithi178=foo.a;count:1 2024-02-14T17:41:57.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:57.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:57.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:56 smithi178 ceph-mon[39811]: pgmap v189: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-02-14T17:41:58.032 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:58 smithi184 ceph-mon[33536]: osdmap e62: 8 total, 8 up, 8 in 2024-02-14T17:41:58.032 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2867139006' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:41:58.032 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:58 smithi184 ceph-mon[33536]: from='client.34358 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:41:58.032 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:58 smithi184 ceph-mon[33536]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:41:58.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:58 smithi178 ceph-mon[39811]: osdmap e62: 8 total, 8 up, 8 in 2024-02-14T17:41:58.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2867139006' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:41:58.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:58 smithi178 ceph-mon[39811]: from='client.34358 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:41:58.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:58 smithi178 ceph-mon[39811]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:41:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:58 smithi178 ceph-mon[30321]: osdmap e62: 8 total, 8 up, 8 in 2024-02-14T17:41:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2867139006' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:41:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:58 smithi178 ceph-mon[30321]: from='client.34358 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:41:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:58 smithi178 ceph-mon[30321]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:41:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[30321]: from='client.34358 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-14T17:41:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[30321]: osdmap e63: 8 total, 8 up, 8 in 2024-02-14T17:41:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[30321]: Checking dashboard <-> RGW credentials 2024-02-14T17:41:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/3919542498' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:41:59.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[30321]: from='client.24391 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:41:59.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[30321]: pgmap v192: 36 pgs: 26 unknown, 10 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:41:59.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[39811]: from='client.34358 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-14T17:41:59.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[39811]: osdmap e63: 8 total, 8 up, 8 in 2024-02-14T17:41:59.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:59.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:59.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:59.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:59.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[39811]: Checking dashboard <-> RGW credentials 2024-02-14T17:41:59.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/3919542498' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:41:59.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[39811]: from='client.24391 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:41:59.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:41:59 smithi178 ceph-mon[39811]: pgmap v192: 36 pgs: 26 unknown, 10 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:41:59.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:59 smithi184 ceph-mon[33536]: from='client.34358 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-14T17:41:59.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:59 smithi184 ceph-mon[33536]: osdmap e63: 8 total, 8 up, 8 in 2024-02-14T17:41:59.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:59 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:59.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:59 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:41:59.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:59 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:59.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:59 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:59.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:59 smithi184 ceph-mon[33536]: Checking dashboard <-> RGW credentials 2024-02-14T17:41:59.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/3919542498' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:41:59.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:59 smithi184 ceph-mon[33536]: from='client.24391 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:41:59.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:41:59 smithi184 ceph-mon[33536]: pgmap v192: 36 pgs: 26 unknown, 10 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[39811]: from='client.24391 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-14T17:42:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[39811]: osdmap e64: 8 total, 8 up, 8 in 2024-02-14T17:42:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[39811]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[39811]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[39811]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:42:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[39811]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:42:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[39811]: osdmap e65: 8 total, 8 up, 8 in 2024-02-14T17:42:00.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[30321]: from='client.24391 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-14T17:42:00.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[30321]: osdmap e64: 8 total, 8 up, 8 in 2024-02-14T17:42:00.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[30321]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[30321]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[30321]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:42:00.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[30321]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:42:00.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:00 smithi178 ceph-mon[30321]: osdmap e65: 8 total, 8 up, 8 in 2024-02-14T17:42:00.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:00 smithi184 ceph-mon[33536]: from='client.24391 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-14T17:42:00.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:00 smithi184 ceph-mon[33536]: osdmap e64: 8 total, 8 up, 8 in 2024-02-14T17:42:00.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:00 smithi184 ceph-mon[33536]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:00 smithi184 ceph-mon[33536]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:42:00.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:00 smithi184 ceph-mon[33536]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:42:00.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:00 smithi184 ceph-mon[33536]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:42:00.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:00 smithi184 ceph-mon[33536]: osdmap e65: 8 total, 8 up, 8 in 2024-02-14T17:42:01.674 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.184 --placement '1;smithi184=iscsi.a' 2024-02-14T17:42:01.920 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:42:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:01 smithi184 ceph-mon[33536]: pgmap v195: 68 pgs: 45 unknown, 23 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-14T17:42:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:01 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:01 smithi184 ceph-mon[33536]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-14T17:42:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:01 smithi184 ceph-mon[33536]: Cluster is now healthy 2024-02-14T17:42:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:01 smithi184 ceph-mon[33536]: osdmap e66: 8 total, 8 up, 8 in 2024-02-14T17:42:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:01 smithi184 ceph-mon[33536]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:01 smithi184 ceph-mon[33536]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[39811]: pgmap v195: 68 pgs: 45 unknown, 23 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-14T17:42:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[39811]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-14T17:42:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[39811]: Cluster is now healthy 2024-02-14T17:42:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[39811]: osdmap e66: 8 total, 8 up, 8 in 2024-02-14T17:42:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[39811]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:02.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[39811]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[30321]: pgmap v195: 68 pgs: 45 unknown, 23 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-14T17:42:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[30321]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-14T17:42:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[30321]: Cluster is now healthy 2024-02-14T17:42:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[30321]: osdmap e66: 8 total, 8 up, 8 in 2024-02-14T17:42:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[30321]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:01 smithi178 ceph-mon[30321]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:42:03.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:03 smithi178 ceph-mon[30321]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:42:03.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:03 smithi178 ceph-mon[30321]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:42:03.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:03 smithi178 ceph-mon[30321]: osdmap e67: 8 total, 8 up, 8 in 2024-02-14T17:42:03.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:03 smithi178 ceph-mon[30321]: pgmap v198: 100 pgs: 8 creating+peering, 55 active+clean, 37 unknown; 578 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:42:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:03 smithi178 ceph-mon[39811]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:42:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:03 smithi178 ceph-mon[39811]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:42:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:03 smithi178 ceph-mon[39811]: osdmap e67: 8 total, 8 up, 8 in 2024-02-14T17:42:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:03 smithi178 ceph-mon[39811]: pgmap v198: 100 pgs: 8 creating+peering, 55 active+clean, 37 unknown; 578 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:42:03.336 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:03 smithi184 ceph-mon[33536]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:42:03.336 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:03 smithi184 ceph-mon[33536]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:42:03.336 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:03 smithi184 ceph-mon[33536]: osdmap e67: 8 total, 8 up, 8 in 2024-02-14T17:42:03.336 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:03 smithi184 ceph-mon[33536]: pgmap v198: 100 pgs: 8 creating+peering, 55 active+clean, 37 unknown; 578 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:42:03.578 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.577+0000 7fb0fe395700 1 -- 172.21.15.184:0/3898629876 >> v1:172.21.15.184:6789/0 conn(0x7fb0f8109ba0 legacy=0x7fb0f810c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:03.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.577+0000 7fb0fe395700 1 -- 172.21.15.184:0/3898629876 shutdown_connections 2024-02-14T17:42:03.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.577+0000 7fb0fe395700 1 -- 172.21.15.184:0/3898629876 >> 172.21.15.184:0/3898629876 conn(0x7fb0f80fbf00 msgr2=0x7fb0f80fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:03.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.577+0000 7fb0fe395700 1 -- 172.21.15.184:0/3898629876 shutdown_connections 2024-02-14T17:42:03.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.577+0000 7fb0fe395700 1 -- 172.21.15.184:0/3898629876 wait complete. 2024-02-14T17:42:03.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.577+0000 7fb0fe395700 1 Processor -- start 2024-02-14T17:42:03.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.577+0000 7fb0fe395700 1 -- start start 2024-02-14T17:42:03.581 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0fe395700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb0f81979e0 con 0x7fb0f81024f0 2024-02-14T17:42:03.581 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0fe395700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb0f81a31d0 con 0x7fb0f8106000 2024-02-14T17:42:03.581 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0fe395700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb0f81a4370 con 0x7fb0f8109ba0 2024-02-14T17:42:03.581 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f77fe700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fb0f8106000 0x7fb0f819d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:49156/0 (socket says 172.21.15.184:49156) 2024-02-14T17:42:03.581 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f77fe700 1 -- 172.21.15.184:0/1184855102 learned_addr learned my addr 172.21.15.184:0/1184855102 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379582805 0 0) 0x7fb0f81a31d0 con 0x7fb0f8106000 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e00035b0 con 0x7fb0f8106000 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212952344 0 0) 0x7fb0f81979e0 con 0x7fb0f81024f0 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f81a31d0 con 0x7fb0f81024f0 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732479237 0 0) 0x7fb0f81a4370 con 0x7fb0f8109ba0 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f81979e0 con 0x7fb0f8109ba0 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1954689955 0 0) 0x7fb0e00035b0 con 0x7fb0f8106000 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb0f81a4370 con 0x7fb0f8106000 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fb0ec0018b0 con 0x7fb0f8106000 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3910407571 0 0) 0x7fb0f81a31d0 con 0x7fb0f81024f0 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb0e00035b0 con 0x7fb0f81024f0 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2477044675 0 0) 0x7fb0f81979e0 con 0x7fb0f8109ba0 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb0f81a31d0 con 0x7fb0f8109ba0 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fb0e8001830 con 0x7fb0f81024f0 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1102938036 0 0) 0x7fb0f81a4370 con 0x7fb0f8106000 2024-02-14T17:42:03.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.578+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 >> v1:172.21.15.178:6790/0 conn(0x7fb0f8109ba0 legacy=0x7fb0f81a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:03.583 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.579+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 >> v1:172.21.15.178:6789/0 conn(0x7fb0f81024f0 legacy=0x7fb0f8196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:03.583 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.579+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f81a5510 con 0x7fb0f8106000 2024-02-14T17:42:03.583 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.579+0000 7fb0fe395700 1 -- 172.21.15.184:0/1184855102 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f81a2220 con 0x7fb0f8106000 2024-02-14T17:42:03.583 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.579+0000 7fb0fe395700 1 -- 172.21.15.184:0/1184855102 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f81a2720 con 0x7fb0f8106000 2024-02-14T17:42:03.583 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.579+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fb0ec001c60 con 0x7fb0f8106000 2024-02-14T17:42:03.583 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.579+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fb0ec005c20 con 0x7fb0f8106000 2024-02-14T17:42:03.583 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.579+0000 7fb0fe395700 1 -- 172.21.15.184:0/1184855102 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e4005150 con 0x7fb0f8106000 2024-02-14T17:42:03.583 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.580+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7fb0ec003950 con 0x7fb0f8106000 2024-02-14T17:42:03.583 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.581+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(68..68 src has 1..68) v4 ==== 5915+0+0 (unknown 1819043654 0 0) 0x7fb0ec0886a0 con 0x7fb0f8106000 2024-02-14T17:42:03.586 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.585+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fb0ec051f40 con 0x7fb0f8106000 2024-02-14T17:42:03.791 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.789+0000 7fb0fe395700 1 -- 172.21.15.184:0/1184855102 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.184", "placement": "1;smithi184=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0e4000c90 con 0x7fb0e006e8d0 2024-02-14T17:42:03.798 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.797+0000 7fb0f57fa700 1 -- 172.21.15.184:0/1184855102 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (unknown 0 0 803663096) 0x7fb0e4000c90 con 0x7fb0e006e8d0 2024-02-14T17:42:03.799 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled iscsi.datapool update... 2024-02-14T17:42:03.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.800+0000 7fb0fe395700 1 -- 172.21.15.184:0/1184855102 >> v1:172.21.15.178:6800/1500761607 conn(0x7fb0e006e8d0 legacy=0x7fb0e0070d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:03.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.800+0000 7fb0fe395700 1 -- 172.21.15.184:0/1184855102 >> v1:172.21.15.184:6789/0 conn(0x7fb0f8106000 legacy=0x7fb0f819d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:03.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.800+0000 7fb0fe395700 1 -- 172.21.15.184:0/1184855102 shutdown_connections 2024-02-14T17:42:03.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.800+0000 7fb0fe395700 1 -- 172.21.15.184:0/1184855102 >> 172.21.15.184:0/1184855102 conn(0x7fb0f80fbf00 msgr2=0x7fb0f80fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:03.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.800+0000 7fb0fe395700 1 -- 172.21.15.184:0/1184855102 shutdown_connections 2024-02-14T17:42:03.801 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:03.800+0000 7fb0fe395700 1 -- 172.21.15.184:0/1184855102 wait complete. 2024-02-14T17:42:04.329 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:04 smithi184 ceph-mon[33536]: osdmap e68: 8 total, 8 up, 8 in 2024-02-14T17:42:04.329 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:04 smithi184 ceph-mon[33536]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:04 smithi184 ceph-mon[33536]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:04 smithi184 ceph-mon[33536]: from='client.24427 v1:172.21.15.184:0/1184855102' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.184", "placement": "1;smithi184=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:04.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:04 smithi184 ceph-mon[33536]: Saving service iscsi.datapool spec with placement smithi184=iscsi.a;count:1 2024-02-14T17:42:04.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:04 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:04.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[30321]: osdmap e68: 8 total, 8 up, 8 in 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[30321]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[30321]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[30321]: from='client.24427 v1:172.21.15.184:0/1184855102' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.184", "placement": "1;smithi184=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[30321]: Saving service iscsi.datapool spec with placement smithi184=iscsi.a;count:1 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[39811]: osdmap e68: 8 total, 8 up, 8 in 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[39811]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[39811]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:42:04.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[39811]: from='client.24427 v1:172.21.15.184:0/1184855102' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.184", "placement": "1;smithi184=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:04.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[39811]: Saving service iscsi.datapool spec with placement smithi184=iscsi.a;count:1 2024-02-14T17:42:04.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:04 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:04.357 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-02-14T17:42:04.357 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:42:04.358 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-14T17:42:04.393 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:42:04.393 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-14T17:42:04.423 DEBUG:teuthology.orchestra.run.smithi184:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@iscsi.iscsi.a.service 2024-02-14T17:42:04.466 INFO:tasks.cephadm:Adding prometheus.a on smithi184 2024-02-14T17:42:04.466 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply prometheus '1;smithi184=a' 2024-02-14T17:42:04.739 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:42:05.167 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:05 smithi184 ceph-mon[33536]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:42:05.167 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:05 smithi184 ceph-mon[33536]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:42:05.167 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:05 smithi184 ceph-mon[33536]: osdmap e69: 8 total, 8 up, 8 in 2024-02-14T17:42:05.167 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.167 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.167 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:05 smithi184 ceph-mon[33536]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.167 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:05 smithi184 ceph-mon[33536]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.167 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:05 smithi184 ceph-mon[33536]: pgmap v201: 132 pgs: 19 creating+peering, 85 active+clean, 28 unknown; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-14T17:42:05.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[30321]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:42:05.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[30321]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:42:05.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[30321]: osdmap e69: 8 total, 8 up, 8 in 2024-02-14T17:42:05.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[30321]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[30321]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[30321]: pgmap v201: 132 pgs: 19 creating+peering, 85 active+clean, 28 unknown; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-14T17:42:05.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[39811]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:42:05.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[39811]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:42:05.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[39811]: osdmap e69: 8 total, 8 up, 8 in 2024-02-14T17:42:05.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2582019531' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/212053341' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[39811]: from='client.34367 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[39811]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:42:05.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:05 smithi178 ceph-mon[39811]: pgmap v201: 132 pgs: 19 creating+peering, 85 active+clean, 28 unknown; 578 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-14T17:42:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[30321]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:42:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[30321]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:42:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[30321]: osdmap e70: 8 total, 8 up, 8 in 2024-02-14T17:42:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-14T17:42:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-14T17:42:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[30321]: Deploying daemon iscsi.iscsi.a on smithi184 2024-02-14T17:42:06.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[39811]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:42:06.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[39811]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:42:06.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[39811]: osdmap e70: 8 total, 8 up, 8 in 2024-02-14T17:42:06.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:06.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:06.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:06.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-14T17:42:06.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-14T17:42:06.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:06.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:06 smithi178 ceph-mon[39811]: Deploying daemon iscsi.iscsi.a on smithi184 2024-02-14T17:42:06.341 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:06 smithi184 ceph-mon[33536]: from='client.34367 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:42:06.341 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:06 smithi184 ceph-mon[33536]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:42:06.341 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:06 smithi184 ceph-mon[33536]: osdmap e70: 8 total, 8 up, 8 in 2024-02-14T17:42:06.341 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:06 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:06.341 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:06 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:06.341 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:06 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:06.341 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:06 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-14T17:42:06.341 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:06 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-14T17:42:06.342 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:06 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:06.342 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:06 smithi184 ceph-mon[33536]: Deploying daemon iscsi.iscsi.a on smithi184 2024-02-14T17:42:06.368 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.339+0000 7f8ae81e3700 1 -- 172.21.15.184:0/74286144 >> v1:172.21.15.184:6789/0 conn(0x7f8ae0108ad0 legacy=0x7f8ae010af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:06.368 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.339+0000 7f8ae81e3700 1 -- 172.21.15.184:0/74286144 shutdown_connections 2024-02-14T17:42:06.368 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.339+0000 7f8ae81e3700 1 -- 172.21.15.184:0/74286144 >> 172.21.15.184:0/74286144 conn(0x7f8ae00fbf00 msgr2=0x7f8ae00fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:06.368 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.339+0000 7f8ae81e3700 1 -- 172.21.15.184:0/74286144 shutdown_connections 2024-02-14T17:42:06.368 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.339+0000 7f8ae81e3700 1 -- 172.21.15.184:0/74286144 wait complete. 2024-02-14T17:42:06.368 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ae81e3700 1 Processor -- start 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ae81e3700 1 -- start start 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ae81e3700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8ae0196920 con 0x7f8ae0108ad0 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ae81e3700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8ae01a2110 con 0x7f8ae0100960 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ae81e3700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8ae01a32b0 con 0x7f8ae0104d90 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ae5f7f700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f8ae0100960 0x7f8ae0195dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:49178/0 (socket says 172.21.15.184:49178) 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ae5f7f700 1 -- 172.21.15.184:0/1979931507 learned_addr learned my addr 172.21.15.184:0/1979931507 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350949389 0 0) 0x7f8ae01a2110 con 0x7f8ae0100960 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8acc0035b0 con 0x7f8ae0100960 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375116734 0 0) 0x7f8ae01a32b0 con 0x7f8ae0104d90 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae01a2110 con 0x7f8ae0104d90 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938219216 0 0) 0x7f8ae0196920 con 0x7f8ae0108ad0 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae01a32b0 con 0x7f8ae0108ad0 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4222104221 0 0) 0x7f8acc0035b0 con 0x7f8ae0100960 2024-02-14T17:42:06.369 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ae0196920 con 0x7f8ae0100960 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.340+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f8ad00018f0 con 0x7f8ae0100960 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2496879069 0 0) 0x7f8ae01a2110 con 0x7f8ae0104d90 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8acc0035b0 con 0x7f8ae0104d90 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3839542335 0 0) 0x7f8ae01a32b0 con 0x7f8ae0108ad0 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ae01a2110 con 0x7f8ae0108ad0 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f8adc0017e0 con 0x7f8ae0104d90 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f8ad8001c80 con 0x7f8ae0108ad0 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1277727222 0 0) 0x7f8ae0196920 con 0x7f8ae0100960 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 >> v1:172.21.15.178:6790/0 conn(0x7f8ae0104d90 legacy=0x7f8ae019c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 >> v1:172.21.15.178:6789/0 conn(0x7f8ae0108ad0 legacy=0x7f8ae019f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae01a4450 con 0x7f8ae0100960 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ae81e3700 1 -- 172.21.15.184:0/1979931507 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae01a34a0 con 0x7f8ae0100960 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ae81e3700 1 -- 172.21.15.184:0/1979931507 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae01a39a0 con 0x7f8ae0100960 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f8ad0001c80 con 0x7f8ae0100960 2024-02-14T17:42:06.370 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.341+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f8ad0005c80 con 0x7f8ae0100960 2024-02-14T17:42:06.371 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.342+0000 7f8ae81e3700 1 -- 172.21.15.184:0/1979931507 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac4005120 con 0x7f8ae0100960 2024-02-14T17:42:06.371 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.342+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7f8ad001c2e0 con 0x7f8ae0100960 2024-02-14T17:42:06.371 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.343+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(70..70 src has 1..70) v4 ==== 5942+0+0 (unknown 2493746336 0 0) 0x7f8ad00887c0 con 0x7f8ae0100960 2024-02-14T17:42:06.371 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.347+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f8ad0052040 con 0x7f8ae0100960 2024-02-14T17:42:06.557 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.555+0000 7f8ae81e3700 1 -- 172.21.15.184:0/1979931507 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ac4000b90 con 0x7f8acc070bf0 2024-02-14T17:42:06.564 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.563+0000 7f8ad6ffd700 1 -- 172.21.15.184:0/1979931507 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (unknown 0 0 1342662408) 0x7f8ac4000b90 con 0x7f8acc070bf0 2024-02-14T17:42:06.564 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled prometheus update... 2024-02-14T17:42:06.566 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.565+0000 7f8ae81e3700 1 -- 172.21.15.184:0/1979931507 >> v1:172.21.15.178:6800/1500761607 conn(0x7f8acc070bf0 legacy=0x7f8acc0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:06.566 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.565+0000 7f8ae81e3700 1 -- 172.21.15.184:0/1979931507 >> v1:172.21.15.184:6789/0 conn(0x7f8ae0100960 legacy=0x7f8ae0195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:06.567 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.565+0000 7f8ae81e3700 1 -- 172.21.15.184:0/1979931507 shutdown_connections 2024-02-14T17:42:06.567 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.565+0000 7f8ae81e3700 1 -- 172.21.15.184:0/1979931507 >> 172.21.15.184:0/1979931507 conn(0x7f8ae00fbf00 msgr2=0x7f8ae010b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:06.567 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.565+0000 7f8ae81e3700 1 -- 172.21.15.184:0/1979931507 shutdown_connections 2024-02-14T17:42:06.567 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:06.565+0000 7f8ae81e3700 1 -- 172.21.15.184:0/1979931507 wait complete. 2024-02-14T17:42:07.715 DEBUG:teuthology.orchestra.run.smithi184:prometheus.a> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@prometheus.a.service 2024-02-14T17:42:07.718 INFO:tasks.cephadm:Adding node-exporter.a on smithi178 2024-02-14T17:42:07.718 INFO:tasks.cephadm:Adding node-exporter.b on smithi184 2024-02-14T17:42:07.718 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply node-exporter '2;smithi178=a;smithi184=b' 2024-02-14T17:42:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:07 smithi178 ceph-mon[30321]: from='client.24448 v1:172.21.15.184:0/1979931507' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:07 smithi178 ceph-mon[30321]: Saving service prometheus spec with placement smithi184=a;count:1 2024-02-14T17:42:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:07 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:07 smithi178 ceph-mon[30321]: pgmap v203: 132 pgs: 19 creating+peering, 103 active+clean, 10 unknown; 579 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1.7 KiB/s wr, 38 op/s 2024-02-14T17:42:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:07 smithi178 ceph-mon[39811]: from='client.24448 v1:172.21.15.184:0/1979931507' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:07 smithi178 ceph-mon[39811]: Saving service prometheus spec with placement smithi184=a;count:1 2024-02-14T17:42:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:07 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:07 smithi178 ceph-mon[39811]: pgmap v203: 132 pgs: 19 creating+peering, 103 active+clean, 10 unknown; 579 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1.7 KiB/s wr, 38 op/s 2024-02-14T17:42:07.861 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:07 smithi184 ceph-mon[33536]: from='client.24448 v1:172.21.15.184:0/1979931507' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:07.861 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:07 smithi184 ceph-mon[33536]: Saving service prometheus spec with placement smithi184=a;count:1 2024-02-14T17:42:07.861 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:07 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:07.861 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:07 smithi184 ceph-mon[33536]: pgmap v203: 132 pgs: 19 creating+peering, 103 active+clean, 10 unknown; 579 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1.7 KiB/s wr, 38 op/s 2024-02-14T17:42:08.328 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:42:09.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:09 smithi184 ceph-mon[33536]: osdmap e71: 8 total, 8 up, 8 in 2024-02-14T17:42:09.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:09 smithi184 ceph-mon[33536]: pgmap v205: 132 pgs: 7 creating+peering, 125 active+clean; 581 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 157 op/s 2024-02-14T17:42:09.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:09 smithi178 ceph-mon[30321]: osdmap e71: 8 total, 8 up, 8 in 2024-02-14T17:42:09.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:09 smithi178 ceph-mon[30321]: pgmap v205: 132 pgs: 7 creating+peering, 125 active+clean; 581 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 157 op/s 2024-02-14T17:42:09.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:09 smithi178 ceph-mon[39811]: osdmap e71: 8 total, 8 up, 8 in 2024-02-14T17:42:09.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:09 smithi178 ceph-mon[39811]: pgmap v205: 132 pgs: 7 creating+peering, 125 active+clean; 581 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 157 op/s 2024-02-14T17:42:10.173 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.169+0000 7efe2055f700 1 -- 172.21.15.184:0/1740110892 >> v1:172.21.15.184:6789/0 conn(0x7efe18109c00 legacy=0x7efe1810c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.170+0000 7efe2055f700 1 -- 172.21.15.184:0/1740110892 shutdown_connections 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.170+0000 7efe2055f700 1 -- 172.21.15.184:0/1740110892 >> 172.21.15.184:0/1740110892 conn(0x7efe180fd070 msgr2=0x7efe180ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.170+0000 7efe2055f700 1 -- 172.21.15.184:0/1740110892 shutdown_connections 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.170+0000 7efe2055f700 1 -- 172.21.15.184:0/1740110892 wait complete. 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.170+0000 7efe2055f700 1 Processor -- start 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.170+0000 7efe2055f700 1 -- start start 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.170+0000 7efe2055f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efe181979d0 con 0x7efe18109c00 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.170+0000 7efe2055f700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efe181a31c0 con 0x7efe18101a90 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.170+0000 7efe2055f700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efe181a4360 con 0x7efe18105ec0 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.170+0000 7efe1e2fb700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7efe18101a90 0x7efe18196e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:54650/0 (socket says 172.21.15.184:54650) 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.170+0000 7efe1e2fb700 1 -- 172.21.15.184:0/820075650 learned_addr learned my addr 172.21.15.184:0/820075650 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618470611 0 0) 0x7efe181a31c0 con 0x7efe18101a90 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe040035b0 con 0x7efe18101a90 2024-02-14T17:42:10.174 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849295180 0 0) 0x7efe181a4360 con 0x7efe18105ec0 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe181a31c0 con 0x7efe18105ec0 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082877021 0 0) 0x7efe181979d0 con 0x7efe18109c00 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe181a4360 con 0x7efe18109c00 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3349900262 0 0) 0x7efe040035b0 con 0x7efe18101a90 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe181979d0 con 0x7efe18101a90 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7efe140018f0 con 0x7efe18101a90 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2422940003 0 0) 0x7efe181a31c0 con 0x7efe18105ec0 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe040035b0 con 0x7efe18105ec0 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7efe08001830 con 0x7efe18105ec0 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 719133502 0 0) 0x7efe181a4360 con 0x7efe18109c00 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe181a31c0 con 0x7efe18109c00 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7efe100049f0 con 0x7efe18109c00 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1803264773 0 0) 0x7efe181979d0 con 0x7efe18101a90 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 >> v1:172.21.15.178:6790/0 conn(0x7efe18105ec0 legacy=0x7efe1819d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:10.175 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 >> v1:172.21.15.178:6789/0 conn(0x7efe18109c00 legacy=0x7efe181a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:10.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe181a5500 con 0x7efe18101a90 2024-02-14T17:42:10.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe2055f700 1 -- 172.21.15.184:0/820075650 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe181a4550 con 0x7efe18101a90 2024-02-14T17:42:10.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe2055f700 1 -- 172.21.15.184:0/820075650 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efe181a4a50 con 0x7efe18101a90 2024-02-14T17:42:10.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7efe14001c80 con 0x7efe18101a90 2024-02-14T17:42:10.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.171+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7efe14005d00 con 0x7efe18101a90 2024-02-14T17:42:10.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.172+0000 7efe2055f700 1 -- 172.21.15.184:0/820075650 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc005150 con 0x7efe18101a90 2024-02-14T17:42:10.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.173+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 15) v1 ==== 91372+0+0 (unknown 2353015468 0 0) 0x7efe14003950 con 0x7efe18101a90 2024-02-14T17:42:10.176 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.173+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 5942+0+0 (unknown 2741107202 0 0) 0x7efe14088830 con 0x7efe18101a90 2024-02-14T17:42:10.179 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.178+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7efe14052090 con 0x7efe18101a90 2024-02-14T17:42:10.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.391+0000 7efe2055f700 1 -- 172.21.15.184:0/820075650 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi178=a;smithi184=b", "target": ["mon-mgr", ""]}) v1 -- 0x7efdfc000bc0 con 0x7efe0406e9f0 2024-02-14T17:42:10.399 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.398+0000 7efe0f7fe700 1 -- 172.21.15.184:0/820075650 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (unknown 0 0 240551134) 0x7efdfc000bc0 con 0x7efe0406e9f0 2024-02-14T17:42:10.400 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled node-exporter update... 2024-02-14T17:42:10.402 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.400+0000 7efe2055f700 1 -- 172.21.15.184:0/820075650 >> v1:172.21.15.178:6800/1500761607 conn(0x7efe0406e9f0 legacy=0x7efe04070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:10.402 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.400+0000 7efe2055f700 1 -- 172.21.15.184:0/820075650 >> v1:172.21.15.184:6789/0 conn(0x7efe18101a90 legacy=0x7efe18196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:10.402 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.400+0000 7efe2055f700 1 -- 172.21.15.184:0/820075650 shutdown_connections 2024-02-14T17:42:10.402 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.400+0000 7efe2055f700 1 -- 172.21.15.184:0/820075650 >> 172.21.15.184:0/820075650 conn(0x7efe180fd070 msgr2=0x7efe1810c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:10.402 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.400+0000 7efe2055f700 1 -- 172.21.15.184:0/820075650 shutdown_connections 2024-02-14T17:42:10.402 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:10.401+0000 7efe2055f700 1 -- 172.21.15.184:0/820075650 wait complete. 2024-02-14T17:42:11.143 DEBUG:teuthology.orchestra.run.smithi178:node-exporter.a> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@node-exporter.a.service 2024-02-14T17:42:11.146 DEBUG:teuthology.orchestra.run.smithi184:node-exporter.b> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@node-exporter.b.service 2024-02-14T17:42:11.148 INFO:tasks.cephadm:Adding alertmanager.a on smithi178 2024-02-14T17:42:11.148 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply alertmanager '1;smithi178=a' 2024-02-14T17:42:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: from='client.24454 v1:172.21.15.184:0/820075650' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi178=a;smithi184=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: Saving service node-exporter spec with placement smithi178=a;smithi184=b;count:2 2024-02-14T17:42:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-14T17:42:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 5.2 KiB/s wr, 166 op/s 2024-02-14T17:42:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:11.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/1839270113' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T17:42:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: from='client.24454 v1:172.21.15.184:0/820075650' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi178=a;smithi184=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: Saving service node-exporter spec with placement smithi178=a;smithi184=b;count:2 2024-02-14T17:42:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-14T17:42:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 5.2 KiB/s wr, 166 op/s 2024-02-14T17:42:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:11.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/1839270113' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T17:42:11.695 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: from='client.24454 v1:172.21.15.184:0/820075650' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi178=a;smithi184=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:11.695 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: Saving service node-exporter spec with placement smithi178=a;smithi184=b;count:2 2024-02-14T17:42:11.695 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.695 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-14T17:42:11.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 5.2 KiB/s wr, 166 op/s 2024-02-14T17:42:11.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:11.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:11.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/1839270113' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T17:42:11.753 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:42:13.799 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:13 smithi184 ceph-mon[33536]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-14T17:42:13.799 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:13 smithi184 ceph-mon[33536]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-02-14T17:42:14.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.023+0000 7fd87077c700 1 -- 172.21.15.184:0/3098949458 >> v1:172.21.15.184:6789/0 conn(0x7fd868101f80 legacy=0x7fd8681023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:14.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.023+0000 7fd87077c700 1 -- 172.21.15.184:0/3098949458 shutdown_connections 2024-02-14T17:42:14.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.023+0000 7fd87077c700 1 -- 172.21.15.184:0/3098949458 >> 172.21.15.184:0/3098949458 conn(0x7fd8680fb690 msgr2=0x7fd8680fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:14.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.023+0000 7fd87077c700 1 -- 172.21.15.184:0/3098949458 shutdown_connections 2024-02-14T17:42:14.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.023+0000 7fd87077c700 1 -- 172.21.15.184:0/3098949458 wait complete. 2024-02-14T17:42:14.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.023+0000 7fd87077c700 1 Processor -- start 2024-02-14T17:42:14.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.023+0000 7fd87077c700 1 -- start start 2024-02-14T17:42:14.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd87077c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd868110b30 con 0x7fd8681016c0 2024-02-14T17:42:14.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd87077c700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd868199d80 con 0x7fd868100b20 2024-02-14T17:42:14.025 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd87077c700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd8681a6520 con 0x7fd868101f80 2024-02-14T17:42:14.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd86e518700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fd868100b20 0x7fd8681991d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:54748/0 (socket says 172.21.15.184:54748) 2024-02-14T17:42:14.026 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd86e518700 1 -- 172.21.15.184:0/739021617 learned_addr learned my addr 172.21.15.184:0/739021617 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435109940 0 0) 0x7fd868199d80 con 0x7fd868100b20 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8540035b0 con 0x7fd868100b20 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889947657 0 0) 0x7fd868110b30 con 0x7fd8681016c0 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd868199d80 con 0x7fd8681016c0 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257854714 0 0) 0x7fd8681a6520 con 0x7fd868101f80 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd868110b30 con 0x7fd868101f80 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2932788777 0 0) 0x7fd8540035b0 con 0x7fd868100b20 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd8681a6520 con 0x7fd868100b20 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd864001880 con 0x7fd868100b20 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1275682947 0 0) 0x7fd868199d80 con 0x7fd8681016c0 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd8540035b0 con 0x7fd8681016c0 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd858001830 con 0x7fd8681016c0 2024-02-14T17:42:14.028 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3238923489 0 0) 0x7fd868110b30 con 0x7fd868101f80 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd868199d80 con 0x7fd868101f80 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.024+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd860001b60 con 0x7fd868101f80 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.025+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2634039915 0 0) 0x7fd8681a6520 con 0x7fd868100b20 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.025+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 >> v1:172.21.15.178:6790/0 conn(0x7fd868101f80 legacy=0x7fd8681a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.025+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 >> v1:172.21.15.178:6789/0 conn(0x7fd8681016c0 legacy=0x7fd86819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.025+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8681a76c0 con 0x7fd868100b20 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.025+0000 7fd87077c700 1 -- 172.21.15.184:0/739021617 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8681a66b0 con 0x7fd868100b20 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.025+0000 7fd87077c700 1 -- 172.21.15.184:0/739021617 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8681a6bb0 con 0x7fd868100b20 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.025+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fd864004790 con 0x7fd868100b20 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.025+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd864005090 con 0x7fd868100b20 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.026+0000 7fd87077c700 1 -- 172.21.15.184:0/739021617 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84c005150 con 0x7fd868100b20 2024-02-14T17:42:14.029 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.026+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 16) v1 ==== 91423+0+0 (unknown 532766425 0 0) 0x7fd8640051f0 con 0x7fd868100b20 2024-02-14T17:42:14.030 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.027+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 5942+0+0 (unknown 2741107202 0 0) 0x7fd864088bf0 con 0x7fd868100b20 2024-02-14T17:42:14.034 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.033+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fd8640522b0 con 0x7fd868100b20 2024-02-14T17:42:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:13 smithi178 ceph-mon[39811]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-14T17:42:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:13 smithi178 ceph-mon[39811]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-02-14T17:42:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:13 smithi178 ceph-mon[30321]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-14T17:42:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:13 smithi178 ceph-mon[30321]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-02-14T17:42:14.274 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.273+0000 7fd87077c700 1 -- 172.21.15.184:0/739021617 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi178=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fd84c000bc0 con 0x7fd85406ea00 2024-02-14T17:42:14.281 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.280+0000 7fd85f7fe700 1 -- 172.21.15.184:0/739021617 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (unknown 0 0 1850065467) 0x7fd84c000bc0 con 0x7fd85406ea00 2024-02-14T17:42:14.281 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled alertmanager update... 2024-02-14T17:42:14.283 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.282+0000 7fd87077c700 1 -- 172.21.15.184:0/739021617 >> v1:172.21.15.178:6800/1500761607 conn(0x7fd85406ea00 legacy=0x7fd854070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:14.283 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.282+0000 7fd87077c700 1 -- 172.21.15.184:0/739021617 >> v1:172.21.15.184:6789/0 conn(0x7fd868100b20 legacy=0x7fd8681991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:14.283 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.282+0000 7fd87077c700 1 -- 172.21.15.184:0/739021617 shutdown_connections 2024-02-14T17:42:14.284 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.282+0000 7fd87077c700 1 -- 172.21.15.184:0/739021617 >> 172.21.15.184:0/739021617 conn(0x7fd8680fb690 msgr2=0x7fd86810e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:14.284 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.282+0000 7fd87077c700 1 -- 172.21.15.184:0/739021617 shutdown_connections 2024-02-14T17:42:14.284 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:14.282+0000 7fd87077c700 1 -- 172.21.15.184:0/739021617 wait complete. 2024-02-14T17:42:15.096 DEBUG:teuthology.orchestra.run.smithi178:alertmanager.a> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@alertmanager.a.service 2024-02-14T17:42:15.099 INFO:tasks.cephadm:Adding grafana.a on smithi184 2024-02-14T17:42:15.099 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply grafana '1;smithi184=a' 2024-02-14T17:42:15.443 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:42:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:15 smithi184 ceph-mon[33536]: from='client.24490 v1:172.21.15.184:0/739021617' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi178=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:15 smithi184 ceph-mon[33536]: Saving service alertmanager spec with placement smithi178=a;count:1 2024-02-14T17:42:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:15 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:15 smithi184 ceph-mon[33536]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 3.6 KiB/s wr, 115 op/s 2024-02-14T17:42:15.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:15 smithi178 ceph-mon[30321]: from='client.24490 v1:172.21.15.184:0/739021617' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi178=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:15.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:15 smithi178 ceph-mon[30321]: Saving service alertmanager spec with placement smithi178=a;count:1 2024-02-14T17:42:15.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:15 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:15.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:15 smithi178 ceph-mon[30321]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 3.6 KiB/s wr, 115 op/s 2024-02-14T17:42:15.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:15 smithi178 ceph-mon[39811]: from='client.24490 v1:172.21.15.184:0/739021617' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi178=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:15.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:15 smithi178 ceph-mon[39811]: Saving service alertmanager spec with placement smithi178=a;count:1 2024-02-14T17:42:15.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:15 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:15.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:15 smithi178 ceph-mon[39811]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 3.6 KiB/s wr, 115 op/s 2024-02-14T17:42:16.668 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:16.668 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:16.668 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:16.668 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:16.668 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:16.668 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:16 smithi184 ceph-mon[33536]: Deploying daemon prometheus.a on smithi184 2024-02-14T17:42:16.668 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:16 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[30321]: Deploying daemon prometheus.a on smithi184 2024-02-14T17:42:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[39811]: Deploying daemon prometheus.a on smithi184 2024-02-14T17:42:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:16 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:17.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.068+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/1718807520 >> v1:172.21.15.184:6789/0 conn(0x7fc0b8102540 legacy=0x7fc0b8104930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:17.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.068+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/1718807520 shutdown_connections 2024-02-14T17:42:17.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.068+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/1718807520 >> 172.21.15.184:0/1718807520 conn(0x7fc0b80fbf50 msgr2=0x7fc0b80fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:17.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.068+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/1718807520 shutdown_connections 2024-02-14T17:42:17.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.068+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/1718807520 wait complete. 2024-02-14T17:42:17.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.068+0000 7fc0bd8b8700 1 Processor -- start 2024-02-14T17:42:17.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bd8b8700 1 -- start start 2024-02-14T17:42:17.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bd8b8700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc0b81978f0 con 0x7fc0b8106050 2024-02-14T17:42:17.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bd8b8700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc0b81a30a0 con 0x7fc0b8102540 2024-02-14T17:42:17.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bd8b8700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc0b81a4240 con 0x7fc0b8109bf0 2024-02-14T17:42:17.072 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0b6ffd700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fc0b8102540 0x7fc0b8196da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:54760/0 (socket says 172.21.15.184:54760) 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0b6ffd700 1 -- 172.21.15.184:0/2545285026 learned_addr learned my addr 172.21.15.184:0/2545285026 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590812103 0 0) 0x7fc0b81a30a0 con 0x7fc0b8102540 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a40035b0 con 0x7fc0b8102540 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277990839 0 0) 0x7fc0b81a4240 con 0x7fc0b8109bf0 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b81a30a0 con 0x7fc0b8109bf0 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727634365 0 0) 0x7fc0b81978f0 con 0x7fc0b8106050 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b81a4240 con 0x7fc0b8106050 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1651923842 0 0) 0x7fc0a40035b0 con 0x7fc0b8102540 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0b81978f0 con 0x7fc0b8102540 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fc0a80017b0 con 0x7fc0b8102540 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.069+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3130055351 0 0) 0x7fc0b81a4240 con 0x7fc0b8106050 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0a40035b0 con 0x7fc0b8106050 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fc0a0001830 con 0x7fc0b8106050 2024-02-14T17:42:17.073 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1672554204 0 0) 0x7fc0b81a30a0 con 0x7fc0b8109bf0 2024-02-14T17:42:17.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0b81a4240 con 0x7fc0b8109bf0 2024-02-14T17:42:17.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2601336258 0 0) 0x7fc0b81978f0 con 0x7fc0b8102540 2024-02-14T17:42:17.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 >> v1:172.21.15.178:6790/0 conn(0x7fc0b8109bf0 legacy=0x7fc0b81a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:17.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 >> v1:172.21.15.178:6789/0 conn(0x7fc0b8106050 legacy=0x7fc0b819d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:17.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b81a53e0 con 0x7fc0b8102540 2024-02-14T17:42:17.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/2545285026 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b81a4430 con 0x7fc0b8102540 2024-02-14T17:42:17.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/2545285026 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b81a4930 con 0x7fc0b8102540 2024-02-14T17:42:17.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fc0a80040a0 con 0x7fc0b8102540 2024-02-14T17:42:17.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.070+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fc0a8005400 con 0x7fc0b8102540 2024-02-14T17:42:17.074 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.071+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/2545285026 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc098005150 con 0x7fc0b8102540 2024-02-14T17:42:17.075 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.071+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 16) v1 ==== 91423+0+0 (unknown 532766425 0 0) 0x7fc0a8004210 con 0x7fc0b8102540 2024-02-14T17:42:17.075 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.072+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 5942+0+0 (unknown 2741107202 0 0) 0x7fc0a8089160 con 0x7fc0b8102540 2024-02-14T17:42:17.077 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.076+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fc0a8052860 con 0x7fc0b8102540 2024-02-14T17:42:17.294 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.292+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/2545285026 --> v1:172.21.15.178:6800/1500761607 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fc098000bc0 con 0x7fc0a406e9c0 2024-02-14T17:42:17.301 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.299+0000 7fc0bc8b6700 1 -- 172.21.15.184:0/2545285026 <== mgr.14150 v1:172.21.15.178:6800/1500761607 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (unknown 0 0 664801700) 0x7fc098000bc0 con 0x7fc0a406e9c0 2024-02-14T17:42:17.301 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled grafana update... 2024-02-14T17:42:17.303 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.302+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/2545285026 >> v1:172.21.15.178:6800/1500761607 conn(0x7fc0a406e9c0 legacy=0x7fc0a4070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:17.303 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.302+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/2545285026 >> v1:172.21.15.184:6789/0 conn(0x7fc0b8102540 legacy=0x7fc0b8196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:17.303 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.302+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/2545285026 shutdown_connections 2024-02-14T17:42:17.303 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.302+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/2545285026 >> 172.21.15.184:0/2545285026 conn(0x7fc0b80fbf50 msgr2=0x7fc0b8104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:17.303 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.302+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/2545285026 shutdown_connections 2024-02-14T17:42:17.303 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:17.302+0000 7fc0bd8b8700 1 -- 172.21.15.184:0/2545285026 wait complete. 2024-02-14T17:42:17.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:17 smithi184 ceph-mon[33536]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 3.1 KiB/s wr, 96 op/s 2024-02-14T17:42:17.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:17 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:17 smithi178 ceph-mon[39811]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 3.1 KiB/s wr, 96 op/s 2024-02-14T17:42:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:17 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:17.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:17 smithi178 ceph-mon[30321]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 3.1 KiB/s wr, 96 op/s 2024-02-14T17:42:17.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:17 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:18.544 DEBUG:teuthology.orchestra.run.smithi184:grafana.a> sudo journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@grafana.a.service 2024-02-14T17:42:18.547 INFO:tasks.cephadm:Setting up client nodes... 2024-02-14T17:42:18.547 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-14T17:42:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:18.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:18 smithi178 ceph-mon[39811]: from='client.24496 v1:172.21.15.184:0/2545285026' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:18 smithi178 ceph-mon[39811]: Saving service grafana spec with placement smithi184=a;count:1 2024-02-14T17:42:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:18 smithi178 ceph-mon[30321]: from='client.24496 v1:172.21.15.184:0/2545285026' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:18 smithi178 ceph-mon[30321]: Saving service grafana spec with placement smithi184=a;count:1 2024-02-14T17:42:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:18 smithi184 ceph-mon[33536]: from='client.24496 v1:172.21.15.184:0/2545285026' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:18 smithi184 ceph-mon[33536]: Saving service grafana spec with placement smithi184=a;count:1 2024-02-14T17:42:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:19 smithi178 ceph-mon[30321]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 1.2 KiB/s wr, 39 op/s 2024-02-14T17:42:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:19 smithi178 ceph-mon[39811]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 1.2 KiB/s wr, 39 op/s 2024-02-14T17:42:19.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:19 smithi184 ceph-mon[33536]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 1.2 KiB/s wr, 39 op/s 2024-02-14T17:42:20.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.530+0000 7f21d36e4700 1 -- 172.21.15.178:0/391145193 >> v1:172.21.15.178:6789/0 conn(0x7f21cc109be0 legacy=0x7f21cc10c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:20.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.530+0000 7f21d36e4700 1 -- 172.21.15.178:0/391145193 shutdown_connections 2024-02-14T17:42:20.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.530+0000 7f21d36e4700 1 -- 172.21.15.178:0/391145193 >> 172.21.15.178:0/391145193 conn(0x7f21cc0fd010 msgr2=0x7f21cc0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:20.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.530+0000 7f21d36e4700 1 -- 172.21.15.178:0/391145193 shutdown_connections 2024-02-14T17:42:20.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.530+0000 7f21d36e4700 1 -- 172.21.15.178:0/391145193 wait complete. 2024-02-14T17:42:20.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.531+0000 7f21d36e4700 1 Processor -- start 2024-02-14T17:42:20.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.531+0000 7f21d36e4700 1 -- start start 2024-02-14T17:42:20.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.531+0000 7f21d36e4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f21cc1976c0 con 0x7f21cc101a70 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.531+0000 7f21d36e4700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f21cc1a2eb0 con 0x7f21cc105ea0 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.531+0000 7f21d36e4700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f21cc1a4050 con 0x7f21cc109be0 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.531+0000 7f21d1480700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f21cc101a70 0x7f21cc196b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:49962/0 (socket says 172.21.15.178:49962) 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.531+0000 7f21d1480700 1 -- 172.21.15.178:0/3921730643 learned_addr learned my addr 172.21.15.178:0/3921730643 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718993275 0 0) 0x7f21cc1976c0 con 0x7f21cc101a70 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21b80035b0 con 0x7f21cc101a70 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183126275 0 0) 0x7f21cc1a4050 con 0x7f21cc109be0 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21cc1976c0 con 0x7f21cc109be0 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776130734 0 0) 0x7f21cc1a2eb0 con 0x7f21cc105ea0 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21cc1a4050 con 0x7f21cc105ea0 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 850980953 0 0) 0x7f21b80035b0 con 0x7f21cc101a70 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21cc1a2eb0 con 0x7f21cc101a70 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f21c8001910 con 0x7f21cc101a70 2024-02-14T17:42:20.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2825183971 0 0) 0x7f21cc1a4050 con 0x7f21cc105ea0 2024-02-14T17:42:20.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21b80035b0 con 0x7f21cc105ea0 2024-02-14T17:42:20.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f21bc0011d0 con 0x7f21cc105ea0 2024-02-14T17:42:20.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2972253041 0 0) 0x7f21cc1976c0 con 0x7f21cc109be0 2024-02-14T17:42:20.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21cc1a4050 con 0x7f21cc109be0 2024-02-14T17:42:20.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f21c4004760 con 0x7f21cc109be0 2024-02-14T17:42:20.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 378393882 0 0) 0x7f21cc1a2eb0 con 0x7f21cc101a70 2024-02-14T17:42:20.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 >> v1:172.21.15.178:6790/0 conn(0x7f21cc109be0 legacy=0x7f21cc1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:20.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 >> v1:172.21.15.184:6789/0 conn(0x7f21cc105ea0 legacy=0x7f21cc19d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:20.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc1a51f0 con 0x7f21cc101a70 2024-02-14T17:42:20.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21d36e4700 1 -- 172.21.15.178:0/3921730643 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc1a4240 con 0x7f21cc101a70 2024-02-14T17:42:20.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.532+0000 7f21d36e4700 1 -- 172.21.15.178:0/3921730643 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc1a4740 con 0x7f21cc101a70 2024-02-14T17:42:20.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.533+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f21c8003b30 con 0x7f21cc101a70 2024-02-14T17:42:20.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.533+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f21c8005c80 con 0x7f21cc101a70 2024-02-14T17:42:20.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.534+0000 7f21d36e4700 1 -- 172.21.15.178:0/3921730643 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b0005150 con 0x7f21cc101a70 2024-02-14T17:42:20.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.534+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 16) v1 ==== 91423+0+0 (unknown 532766425 0 0) 0x7f21c801c310 con 0x7f21cc101a70 2024-02-14T17:42:20.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.534+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 5942+0+0 (unknown 2741107202 0 0) 0x7f21c8088800 con 0x7f21cc101a70 2024-02-14T17:42:20.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.539+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f21c8052c10 con 0x7f21cc101a70 2024-02-14T17:42:20.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.808+0000 7f21d36e4700 1 -- 172.21.15.178:0/3921730643 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f21b0005400 con 0x7f21cc101a70 2024-02-14T17:42:20.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.810+0000 7f21c27fc700 1 -- 172.21.15.178:0/3921730643 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (unknown 57899324 0 2934455727) 0x7f21c8054d10 con 0x7f21cc101a70 2024-02-14T17:42:20.812 INFO:teuthology.orchestra.run.smithi178.stdout:[client.0] 2024-02-14T17:42:20.812 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQB8+8xlm9FFMBAAdrLKThfcPabT8/L8p8eckA== 2024-02-14T17:42:20.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.812+0000 7f21d36e4700 1 -- 172.21.15.178:0/3921730643 >> v1:172.21.15.178:6800/1500761607 conn(0x7f21b806ed00 legacy=0x7f21b80711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:20.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.812+0000 7f21d36e4700 1 -- 172.21.15.178:0/3921730643 >> v1:172.21.15.178:6789/0 conn(0x7f21cc101a70 legacy=0x7f21cc196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:20.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.812+0000 7f21d36e4700 1 -- 172.21.15.178:0/3921730643 shutdown_connections 2024-02-14T17:42:20.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.812+0000 7f21d36e4700 1 -- 172.21.15.178:0/3921730643 >> 172.21.15.178:0/3921730643 conn(0x7f21cc0fd010 msgr2=0x7f21cc10ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:20.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.812+0000 7f21d36e4700 1 -- 172.21.15.178:0/3921730643 shutdown_connections 2024-02-14T17:42:20.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:20.813+0000 7f21d36e4700 1 -- 172.21.15.178:0/3921730643 wait complete. 2024-02-14T17:42:21.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:20 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-02-14T17:42:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:20 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2024-02-14T17:42:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:20 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-02-14T17:42:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3921730643' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:42:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3921730643' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:42:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:20 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-02-14T17:42:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:20 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2024-02-14T17:42:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:20 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-02-14T17:42:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3921730643' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:42:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3921730643' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:42:21.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:20 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-02-14T17:42:21.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:20 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2024-02-14T17:42:21.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:20 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-02-14T17:42:21.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3921730643' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:42:21.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3921730643' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:42:21.661 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:42:21.661 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-14T17:42:21.661 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-14T17:42:21.705 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-14T17:42:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[39811]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 8.7 KiB/s rd, 767 B/s wr, 20 op/s 2024-02-14T17:42:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-02-14T17:42:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]': finished 2024-02-14T17:42:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-02-14T17:42:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[39811]: osdmap e72: 8 total, 8 up, 8 in 2024-02-14T17:42:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[39811]: osdmap e73: 8 total, 8 up, 8 in 2024-02-14T17:42:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[30321]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 8.7 KiB/s rd, 767 B/s wr, 20 op/s 2024-02-14T17:42:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-02-14T17:42:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]': finished 2024-02-14T17:42:22.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-02-14T17:42:22.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:22.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[30321]: osdmap e72: 8 total, 8 up, 8 in 2024-02-14T17:42:22.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:21 smithi178 ceph-mon[30321]: osdmap e73: 8 total, 8 up, 8 in 2024-02-14T17:42:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:21 smithi184 ceph-mon[33536]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 8.7 KiB/s rd, 767 B/s wr, 20 op/s 2024-02-14T17:42:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:21 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-02-14T17:42:22.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:21 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]': finished 2024-02-14T17:42:22.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:21 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-02-14T17:42:22.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:21 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:22.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:21 smithi184 ceph-mon[33536]: osdmap e72: 8 total, 8 up, 8 in 2024-02-14T17:42:22.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:21 smithi184 ceph-mon[33536]: osdmap e73: 8 total, 8 up, 8 in 2024-02-14T17:42:23.192 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:22 smithi184 ceph-mon[33536]: Health check failed: Degraded data redundancy: 4/594 objects degraded (0.673%), 1 pg degraded (PG_DEGRADED) 2024-02-14T17:42:23.227 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:42:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:22 smithi178 ceph-mon[39811]: Health check failed: Degraded data redundancy: 4/594 objects degraded (0.673%), 1 pg degraded (PG_DEGRADED) 2024-02-14T17:42:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:22 smithi178 ceph-mon[30321]: Health check failed: Degraded data redundancy: 4/594 objects degraded (0.673%), 1 pg degraded (PG_DEGRADED) 2024-02-14T17:42:24.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:23 smithi184 ceph-mon[33536]: pgmap v214: 132 pgs: 1 activating+degraded, 131 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s; 4/594 objects degraded (0.673%) 2024-02-14T17:42:24.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:23 smithi178 ceph-mon[39811]: pgmap v214: 132 pgs: 1 activating+degraded, 131 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s; 4/594 objects degraded (0.673%) 2024-02-14T17:42:24.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:23 smithi178 ceph-mon[30321]: pgmap v214: 132 pgs: 1 activating+degraded, 131 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s; 4/594 objects degraded (0.673%) 2024-02-14T17:42:26.098 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:25 smithi184 ceph-mon[33536]: pgmap v215: 132 pgs: 1 activating+degraded, 131 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 4/594 objects degraded (0.673%); 41 B/s, 1 objects/s recovering 2024-02-14T17:42:26.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:25 smithi178 ceph-mon[39811]: pgmap v215: 132 pgs: 1 activating+degraded, 131 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 4/594 objects degraded (0.673%); 41 B/s, 1 objects/s recovering 2024-02-14T17:42:26.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:25 smithi178 ceph-mon[30321]: pgmap v215: 132 pgs: 1 activating+degraded, 131 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 4/594 objects degraded (0.673%); 41 B/s, 1 objects/s recovering 2024-02-14T17:42:27.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.919+0000 7f68d9581700 1 -- 172.21.15.184:0/278481416 >> v1:172.21.15.184:6789/0 conn(0x7f68d4100950 legacy=0x7f68d4100d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:27.924 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.919+0000 7f68d9581700 1 -- 172.21.15.184:0/278481416 shutdown_connections 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.919+0000 7f68d9581700 1 -- 172.21.15.184:0/278481416 >> 172.21.15.184:0/278481416 conn(0x7f68d40fbf30 msgr2=0x7f68d40fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.919+0000 7f68d9581700 1 -- 172.21.15.184:0/278481416 shutdown_connections 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.919+0000 7f68d9581700 1 -- 172.21.15.184:0/278481416 wait complete. 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.919+0000 7f68d9581700 1 Processor -- start 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68d9581700 1 -- start start 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68d9581700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f68d410fbd0 con 0x7f68d4108ac0 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68d9581700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f68d41979d0 con 0x7f68d4100950 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68d9581700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f68d41a4170 con 0x7f68d4104d80 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68d2ffd700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f68d4100950 0x7f68d4196e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:40074/0 (socket says 172.21.15.184:40074) 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68d2ffd700 1 -- 172.21.15.184:0/3185866232 learned_addr learned my addr 172.21.15.184:0/3185866232 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440617028 0 0) 0x7f68d41979d0 con 0x7f68d4100950 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68bc0035b0 con 0x7f68d4100950 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739166795 0 0) 0x7f68d41a4170 con 0x7f68d4104d80 2024-02-14T17:42:27.925 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d41979d0 con 0x7f68d4104d80 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845537428 0 0) 0x7f68d410fbd0 con 0x7f68d4108ac0 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d41a4170 con 0x7f68d4108ac0 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1055451196 0 0) 0x7f68bc0035b0 con 0x7f68d4100950 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68d410fbd0 con 0x7f68d4100950 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f68c4001370 con 0x7f68d4100950 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3658859078 0 0) 0x7f68d41a4170 con 0x7f68d4108ac0 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68bc0035b0 con 0x7f68d4108ac0 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 701715111 0 0) 0x7f68d41979d0 con 0x7f68d4104d80 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68d41a4170 con 0x7f68d4104d80 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.920+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f68cc001af0 con 0x7f68d4108ac0 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.921+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f68c80017b0 con 0x7f68d4104d80 2024-02-14T17:42:27.926 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.921+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1796008803 0 0) 0x7f68d410fbd0 con 0x7f68d4100950 2024-02-14T17:42:27.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.921+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 >> v1:172.21.15.178:6790/0 conn(0x7f68d4104d80 legacy=0x7f68d419d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:27.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.921+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 >> v1:172.21.15.178:6789/0 conn(0x7f68d4108ac0 legacy=0x7f68d41a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:27.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.921+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d41a5310 con 0x7f68d4100950 2024-02-14T17:42:27.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.921+0000 7f68d9581700 1 -- 172.21.15.184:0/3185866232 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d41a4360 con 0x7f68d4100950 2024-02-14T17:42:27.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.921+0000 7f68d9581700 1 -- 172.21.15.184:0/3185866232 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d41a4860 con 0x7f68d4100950 2024-02-14T17:42:27.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.921+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f68c4001af0 con 0x7f68d4100950 2024-02-14T17:42:27.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.921+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f68c4005030 con 0x7f68d4100950 2024-02-14T17:42:27.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.922+0000 7f68d9581700 1 -- 172.21.15.184:0/3185866232 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c0005150 con 0x7f68d4100950 2024-02-14T17:42:27.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.922+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 16) v1 ==== 91423+0+0 (unknown 532766425 0 0) 0x7f68c4005230 con 0x7f68d4100950 2024-02-14T17:42:27.927 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.923+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6001+0+0 (unknown 2253613790 0 0) 0x7f68c4088aa0 con 0x7f68d4100950 2024-02-14T17:42:27.929 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:27.927+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f68c40521e0 con 0x7f68d4100950 2024-02-14T17:42:28.204 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:28.201+0000 7f68d9581700 1 -- 172.21.15.184:0/3185866232 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f68c0005400 con 0x7f68d4100950 2024-02-14T17:42:28.204 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:27 smithi184 ceph-mon[33536]: pgmap v216: 132 pgs: 1 activating+degraded, 131 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 4/594 objects degraded (0.673%); 41 B/s, 1 objects/s recovering 2024-02-14T17:42:28.208 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:28.206+0000 7f68bbfff700 1 -- 172.21.15.184:0/3185866232 <== mon.1 v1:172.21.15.184:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) v1 ==== 170+0+59 (unknown 3508017507 0 3226082025) 0x7f68c4054110 con 0x7f68d4100950 2024-02-14T17:42:28.208 INFO:teuthology.orchestra.run.smithi184.stdout:[client.1] 2024-02-14T17:42:28.209 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQCE+8xlhF8sDBAA7SQJvy0N6QJ70FIZ2Eg7pg== 2024-02-14T17:42:28.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:28.208+0000 7f68d9581700 1 -- 172.21.15.184:0/3185866232 >> v1:172.21.15.178:6800/1500761607 conn(0x7f68bc06eb20 legacy=0x7f68bc070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:28.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:28.208+0000 7f68d9581700 1 -- 172.21.15.184:0/3185866232 >> v1:172.21.15.184:6789/0 conn(0x7f68d4100950 legacy=0x7f68d4196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:28.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:28.208+0000 7f68d9581700 1 -- 172.21.15.184:0/3185866232 shutdown_connections 2024-02-14T17:42:28.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:28.208+0000 7f68d9581700 1 -- 172.21.15.184:0/3185866232 >> 172.21.15.184:0/3185866232 conn(0x7f68d40fbf30 msgr2=0x7f68d4069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:28.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:28.208+0000 7f68d9581700 1 -- 172.21.15.184:0/3185866232 shutdown_connections 2024-02-14T17:42:28.211 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-14T17:42:28.208+0000 7f68d9581700 1 -- 172.21.15.184:0/3185866232 wait complete. 2024-02-14T17:42:28.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:27 smithi178 ceph-mon[39811]: pgmap v216: 132 pgs: 1 activating+degraded, 131 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 4/594 objects degraded (0.673%); 41 B/s, 1 objects/s recovering 2024-02-14T17:42:28.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:27 smithi178 ceph-mon[30321]: pgmap v216: 132 pgs: 1 activating+degraded, 131 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 4/594 objects degraded (0.673%); 41 B/s, 1 objects/s recovering 2024-02-14T17:42:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.184:0/3185866232' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:42:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:28 smithi184 ceph-mon[33536]: from='client.24508 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:42:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:28 smithi184 ceph-mon[33536]: from='client.24508 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:42:29.220 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:28 smithi184 systemd[1]: Starting Ceph prometheus.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:42:29.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.184:0/3185866232' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:42:29.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:28 smithi178 ceph-mon[39811]: from='client.24508 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:42:29.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:28 smithi178 ceph-mon[39811]: from='client.24508 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:42:29.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.184:0/3185866232' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:42:29.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:28 smithi178 ceph-mon[30321]: from='client.24508 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:42:29.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:28 smithi178 ceph-mon[30321]: from='client.24508 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:42:30.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:29 smithi184 ceph-mon[33536]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 54 B/s, 2 objects/s recovering 2024-02-14T17:42:30.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:29 smithi184 ceph-mon[33536]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/594 objects degraded (0.673%), 1 pg degraded) 2024-02-14T17:42:30.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:29 smithi184 ceph-mon[33536]: Cluster is now healthy 2024-02-14T17:42:30.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:29 smithi178 ceph-mon[39811]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 54 B/s, 2 objects/s recovering 2024-02-14T17:42:30.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:29 smithi178 ceph-mon[39811]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/594 objects degraded (0.673%), 1 pg degraded) 2024-02-14T17:42:30.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:29 smithi178 ceph-mon[39811]: Cluster is now healthy 2024-02-14T17:42:30.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:29 smithi178 ceph-mon[30321]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 54 B/s, 2 objects/s recovering 2024-02-14T17:42:30.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:29 smithi178 ceph-mon[30321]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/594 objects degraded (0.673%), 1 pg degraded) 2024-02-14T17:42:30.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:29 smithi178 ceph-mon[30321]: Cluster is now healthy 2024-02-14T17:42:30.337 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-14T17:42:30.337 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-14T17:42:30.337 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-14T17:42:30.385 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-14T17:42:30.385 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-14T17:42:30.385 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph mgr dump --format=json 2024-02-14T17:42:30.634 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:30.833 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:30 smithi184 podman[64697]: 2024-02-14 17:42:30.467541622 +0000 UTC m=+0.018777848 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-14T17:42:30.833 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:30 smithi184 podman[64697]: 2024-02-14 17:42:30.746875761 +0000 UTC m=+0.298111978 container create 52997012da916155824f86293051dbb0eeab547f947af108b68f5528d309d39b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:42:31.100 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 podman[64697]: 2024-02-14 17:42:31.015841417 +0000 UTC m=+0.567077634 container init 52997012da916155824f86293051dbb0eeab547f947af108b68f5528d309d39b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:42:31.100 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 podman[64697]: 2024-02-14 17:42:31.021727599 +0000 UTC m=+0.572963820 container start 52997012da916155824f86293051dbb0eeab547f947af108b68f5528d309d39b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:42:31.100 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.046Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-14T17:42:31.100 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.046Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-14T17:42:31.100 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.047Z caller=main.go:570 level=info host_details="(Linux 5.14.0-420.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 9 18:10:28 UTC 2024 x86_64 smithi184 (none))" 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.047Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.047Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.048Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.048Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.050Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.050Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.051Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.051Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.574µs 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.051Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.052Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.052Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=18.041µs wal_replay_duration=166.109µs wbl_replay_duration=115ns total_replay_duration=201.836µs 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.053Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-14T17:42:31.101 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.053Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-14T17:42:31.102 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.053Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-14T17:42:31.102 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.071Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=17.992801ms db_storage=819ns remote_storage=1.08µs web_handler=389ns query_engine=966ns scrape=399.269µs scrape_sd=78.496µs notify=1.193µs notify_sd=1.984µs rules=17.096855ms tracing=24.229µs 2024-02-14T17:42:31.102 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.071Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-14T17:42:31.102 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:42:31.071Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-14T17:42:31.102 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 bash[64697]: 52997012da916155824f86293051dbb0eeab547f947af108b68f5528d309d39b 2024-02-14T17:42:31.469 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:42:31 smithi184 systemd[1]: Started Ceph prometheus.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:42:32.166 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[39811]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 44 B/s, 1 objects/s recovering 2024-02-14T17:42:32.166 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:32.166 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:32.166 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:32.166 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:32.166 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-14T17:42:32.166 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[30321]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 44 B/s, 1 objects/s recovering 2024-02-14T17:42:32.167 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:32.167 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:32.167 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:32.167 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:32.167 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:31 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-14T17:42:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-mon[33536]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 44 B/s, 1 objects/s recovering 2024-02-14T17:42:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' 2024-02-14T17:42:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:31 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-14T17:42:32.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.404+0000 7fc292a75700 1 -- 172.21.15.178:0/1846458432 >> v1:172.21.15.178:6790/0 conn(0x7fc28c10a8f0 legacy=0x7fc28c10cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:32.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.404+0000 7fc292a75700 1 -- 172.21.15.178:0/1846458432 shutdown_connections 2024-02-14T17:42:32.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.404+0000 7fc292a75700 1 -- 172.21.15.178:0/1846458432 >> 172.21.15.178:0/1846458432 conn(0x7fc28c0fd550 msgr2=0x7fc28c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:32.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.404+0000 7fc292a75700 1 -- 172.21.15.178:0/1846458432 shutdown_connections 2024-02-14T17:42:32.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.404+0000 7fc292a75700 1 -- 172.21.15.178:0/1846458432 wait complete. 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.405+0000 7fc292a75700 1 Processor -- start 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.405+0000 7fc292a75700 1 -- start start 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.405+0000 7fc292a75700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc28c1a8b40 con 0x7fc28c10e4c0 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.405+0000 7fc292a75700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc28c1a9ce0 con 0x7fc28c107a40 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.405+0000 7fc292a75700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc28c1aaec0 con 0x7fc28c10a8f0 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.405+0000 7fc28bfff700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7fc28c10a8f0 0x7fc28c1a3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:38376/0 (socket says 172.21.15.178:38376) 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.405+0000 7fc290811700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fc28c107a40 0x7fc28c075860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.178:35682/0 (socket says 172.21.15.178:35682) 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.405+0000 7fc28bfff700 1 -- 172.21.15.178:0/3471283393 learned_addr learned my addr 172.21.15.178:0/3471283393 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878627695 0 0) 0x7fc28c1aaec0 con 0x7fc28c10a8f0 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc27c0035b0 con 0x7fc28c10a8f0 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035866052 0 0) 0x7fc28c1a9ce0 con 0x7fc28c107a40 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc28c1aaec0 con 0x7fc28c107a40 2024-02-14T17:42:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2890914067 0 0) 0x7fc28c1aaec0 con 0x7fc28c107a40 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc28c1a9ce0 con 0x7fc28c107a40 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fc2800012d0 con 0x7fc28c107a40 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3162577565 0 0) 0x7fc27c0035b0 con 0x7fc28c10a8f0 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc28c1aaec0 con 0x7fc28c10a8f0 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203276749 0 0) 0x7fc28c1a8b40 con 0x7fc28c10e4c0 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc27c0035b0 con 0x7fc28c10e4c0 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fc278004400 con 0x7fc28c10a8f0 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 361723339 0 0) 0x7fc28c1a9ce0 con 0x7fc28c107a40 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 >> v1:172.21.15.178:6790/0 conn(0x7fc28c10a8f0 legacy=0x7fc28c1a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 >> v1:172.21.15.178:6789/0 conn(0x7fc28c10e4c0 legacy=0x7fc28c1a7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c1ac0e0 con 0x7fc28c107a40 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc292a75700 1 -- 172.21.15.178:0/3471283393 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28c1a9ed0 con 0x7fc28c107a40 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fc280001580 con 0x7fc28c107a40 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fc280005c00 con 0x7fc28c107a40 2024-02-14T17:42:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.406+0000 7fc292a75700 1 -- 172.21.15.178:0/3471283393 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28c1aa350 con 0x7fc28c107a40 2024-02-14T17:42:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.407+0000 7fc292a75700 1 -- 172.21.15.178:0/3471283393 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28c103180 con 0x7fc28c107a40 2024-02-14T17:42:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.408+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 17) v1 ==== 91437+0+0 (unknown 2512001928 0 0) 0x7fc28001c2c0 con 0x7fc28c107a40 2024-02-14T17:42:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.408+0000 7fc28bfff700 1 -- 172.21.15.178:0/3471283393 >> v1:172.21.15.178:6800/1500761607 conn(0x7fc27c06e9e0 legacy=0x7fc27c070ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1500761607 2024-02-14T17:42:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.408+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6001+0+0 (unknown 2253613790 0 0) 0x7fc280088a70 con 0x7fc28c107a40 2024-02-14T17:42:32.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.412+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fc28008ae60 con 0x7fc28c107a40 2024-02-14T17:42:32.582 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:32 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ignoring --setuser ceph since I am not root 2024-02-14T17:42:32.583 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:32 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ignoring --setgroup ceph since I am not root 2024-02-14T17:42:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.609+0000 7fc28bfff700 1 -- 172.21.15.178:0/3471283393 >> v1:172.21.15.178:6800/1500761607 conn(0x7fc27c06e9e0 legacy=0x7fc27c070ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1500761607 2024-02-14T17:42:32.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.671+0000 7fc292a75700 1 -- 172.21.15.178:0/3471283393 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc28c068560 con 0x7fc28c107a40 2024-02-14T17:42:32.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.675+0000 7fc289ffb700 1 -- 172.21.15.178:0/3471283393 <== mon.1 v1:172.21.15.184:6789/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+174820 (unknown 831476310 0 3992948734) 0x7fc28008ae60 con 0x7fc28c107a40 2024-02-14T17:42:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:42:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.678+0000 7fc292a75700 1 -- 172.21.15.178:0/3471283393 >> v1:172.21.15.178:6800/1500761607 conn(0x7fc27c06e9e0 legacy=0x7fc27c070ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:42:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.678+0000 7fc292a75700 1 -- 172.21.15.178:0/3471283393 >> v1:172.21.15.184:6789/0 conn(0x7fc28c107a40 legacy=0x7fc28c075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.678+0000 7fc292a75700 1 -- 172.21.15.178:0/3471283393 shutdown_connections 2024-02-14T17:42:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.678+0000 7fc292a75700 1 -- 172.21.15.178:0/3471283393 >> 172.21.15.178:0/3471283393 conn(0x7fc28c0fd550 msgr2=0x7fc28c111130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.678+0000 7fc292a75700 1 -- 172.21.15.178:0/3471283393 shutdown_connections 2024-02-14T17:42:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:32.678+0000 7fc292a75700 1 -- 172.21.15.178:0/3471283393 wait complete. 2024-02-14T17:42:32.719 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:32 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: ignoring --setuser ceph since I am not root 2024-02-14T17:42:32.720 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:32 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: ignoring --setgroup ceph since I am not root 2024-02-14T17:42:32.884 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:32 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:32.787+0000 7f2dddc7c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:42:33.206 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:32 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:32.764+0000 7fcf2940d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:42:33.206 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:32 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:32.857+0000 7fcf2940d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:42:33.206 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:32 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:32.882+0000 7f2dddc7c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:42:33.443 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6800","nonce":1500761607}]},"active_addr":"172.21.15.178:6800/1500761607","active_change":"2024-02-14T17:37:20.729926+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.178:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":847383463}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":1798510550}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":1973091493}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":3583216299}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":1784706701}]}]} 2024-02-14T17:42:33.447 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-14T17:42:33.447 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-14T17:42:33.448 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T17:42:33.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:33 smithi184 ceph-mon[33536]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-14T17:42:33.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:33 smithi184 ceph-mon[33536]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-14T17:42:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3471283393' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T17:42:33.472 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:33 smithi178 ceph-mon[39811]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-14T17:42:33.473 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:33 smithi178 ceph-mon[39811]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-14T17:42:33.473 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3471283393' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T17:42:33.473 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:33 smithi178 ceph-mon[30321]: from='mgr.14150 v1:172.21.15.178:0/4227278927' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-14T17:42:33.473 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:33 smithi178 ceph-mon[30321]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-14T17:42:33.473 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3471283393' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T17:42:33.692 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:33.794 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:33 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:33.540+0000 7f2dddc7c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:42:33.795 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:33 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:33.639+0000 7f2dddc7c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:42:33.795 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:33 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:33.792+0000 7f2dddc7c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:42:33.818 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:33 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:33.489+0000 7fcf2940d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:42:33.818 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:33 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:33.582+0000 7fcf2940d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:42:33.818 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:33 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:33.722+0000 7fcf2940d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:42:33.818 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:33 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:33.816+0000 7fcf2940d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:42:34.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:33 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:33.892+0000 7f2dddc7c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:42:34.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:33 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:33.987+0000 7f2dddc7c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:42:34.166 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:33 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:33.899+0000 7fcf2940d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:42:34.166 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:34 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:34.163+0000 7fcf2940d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:42:34.376 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:34 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:34.265+0000 7f2dddc7c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:42:34.432 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:34 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:34.278+0000 7fcf2940d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:42:34.433 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:34 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:34.366+0000 7fcf2940d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:42:34.719 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:34 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:34.467+0000 7fcf2940d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:42:34.719 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:34 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:34.652+0000 7fcf2940d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:42:34.741 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:34 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:34.374+0000 7f2dddc7c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:42:34.741 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:34 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:34.461+0000 7f2dddc7c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:42:34.741 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:34 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:34.559+0000 7f2dddc7c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:42:35.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:34 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:34.739+0000 7f2dddc7c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:42:35.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.488+0000 7f0b11951700 1 -- 172.21.15.178:0/3183450813 >> v1:172.21.15.178:6790/0 conn(0x7f0b0c107a00 legacy=0x7f0b0c107de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:35.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.488+0000 7f0b11951700 1 -- 172.21.15.178:0/3183450813 shutdown_connections 2024-02-14T17:42:35.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.488+0000 7f0b11951700 1 -- 172.21.15.178:0/3183450813 >> 172.21.15.178:0/3183450813 conn(0x7f0b0c0fd530 msgr2=0x7f0b0c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:35.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.488+0000 7f0b11951700 1 -- 172.21.15.178:0/3183450813 shutdown_connections 2024-02-14T17:42:35.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.488+0000 7f0b11951700 1 -- 172.21.15.178:0/3183450813 wait complete. 2024-02-14T17:42:35.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.488+0000 7f0b11951700 1 Processor -- start 2024-02-14T17:42:35.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.488+0000 7f0b11951700 1 -- start start 2024-02-14T17:42:35.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b11951700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0b0c1a6970 con 0x7f0b0c10e480 2024-02-14T17:42:35.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b11951700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0b0c1a7b70 con 0x7f0b0c107a00 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b11951700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0b0c1a8d70 con 0x7f0b0c10a8b0 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b0a7fc700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f0b0c10a8b0 0x7f0b0c1a1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:40316/0 (socket says 172.21.15.178:40316) 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b0b7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f0b0c10e480 0x7f0b0c1a5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:35928/0 (socket says 172.21.15.178:35928) 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b0a7fc700 1 -- 172.21.15.178:0/3587223187 learned_addr learned my addr 172.21.15.178:0/3587223187 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566591781 0 0) 0x7f0b0c1a8d70 con 0x7f0b0c10a8b0 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0af40035b0 con 0x7f0b0c10a8b0 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613006521 0 0) 0x7f0b0c1a6970 con 0x7f0b0c10e480 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b0c1a8d70 con 0x7f0b0c10e480 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884692711 0 0) 0x7f0b0c1a7b70 con 0x7f0b0c107a00 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b0c1a6970 con 0x7f0b0c107a00 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 95297426 0 0) 0x7f0af40035b0 con 0x7f0b0c10a8b0 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0b0c1a7b70 con 0x7f0b0c10a8b0 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f0b000018b0 con 0x7f0b0c10a8b0 2024-02-14T17:42:35.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3228602664 0 0) 0x7f0b0c1a6970 con 0x7f0b0c107a00 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0af40035b0 con 0x7f0b0c107a00 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1470419430 0 0) 0x7f0b0c1a8d70 con 0x7f0b0c10e480 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0b0c1a6970 con 0x7f0b0c10e480 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f0afc0043e0 con 0x7f0b0c107a00 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f0b040014b0 con 0x7f0b0c10e480 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.489+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1587174174 0 0) 0x7f0af40035b0 con 0x7f0b0c107a00 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.490+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 >> v1:172.21.15.178:6790/0 conn(0x7f0b0c10a8b0 legacy=0x7f0b0c1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.490+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 >> v1:172.21.15.178:6789/0 conn(0x7f0b0c10e480 legacy=0x7f0b0c1a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.490+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c1a9f70 con 0x7f0b0c107a00 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.490+0000 7f0b11951700 1 -- 172.21.15.178:0/3587223187 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c1a8f60 con 0x7f0b0c107a00 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.490+0000 7f0b11951700 1 -- 172.21.15.178:0/3587223187 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c1a93e0 con 0x7f0b0c107a00 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.490+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f0afc001ab0 con 0x7f0b0c107a00 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.490+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f0afc004cf0 con 0x7f0b0c107a00 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.490+0000 7f0b11951700 1 -- 172.21.15.178:0/3587223187 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af8005120 con 0x7f0b0c107a00 2024-02-14T17:42:35.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.491+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 17) v1 ==== 91437+0+0 (unknown 2512001928 0 0) 0x7f0afc0037f0 con 0x7f0b0c107a00 2024-02-14T17:42:35.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.491+0000 7f0b0a7fc700 1 -- 172.21.15.178:0/3587223187 >> v1:172.21.15.178:6800/1500761607 conn(0x7f0af4073030 legacy=0x7f0af40754f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1500761607 2024-02-14T17:42:35.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.492+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6001+0+0 (unknown 2253613790 0 0) 0x7f0afc087850 con 0x7f0b0c107a00 2024-02-14T17:42:35.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.496+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f0afc050ed0 con 0x7f0b0c107a00 2024-02-14T17:42:35.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.691+0000 7f0b11951700 1 -- 172.21.15.178:0/3587223187 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0af80053d0 con 0x7f0b0c107a00 2024-02-14T17:42:35.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.693+0000 7f0b0a7fc700 1 -- 172.21.15.178:0/3587223187 >> v1:172.21.15.178:6800/1500761607 conn(0x7f0af4073030 legacy=0x7f0af40754f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1500761607 2024-02-14T17:42:35.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.693+0000 7f0b1094f700 1 -- 172.21.15.178:0/3587223187 <== mon.1 v1:172.21.15.184:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+20711 (unknown 2590290900 0 1209335437) 0x7f0afc052e00 con 0x7f0b0c107a00 2024-02-14T17:42:35.695 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:42:35.695 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":73,"fsid":"6e1588ec-cb5f-11ee-95b9-87774f69a715","created":"2024-02-14T17:36:20.333938+0000","modified":"2024-02-14T17:42:21.816873+0000","last_up_change":"2024-02-14T17:41:44.153032+0000","last_in_change":"2024-02-14T17:41:26.094057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T17:39:39.062242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-14T17:41:55.967442+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-14T17:41:56.498770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-14T17:41:58.450521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-14T17:42:00.289855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-14T17:42:02.364674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96b590dc-522b-4b8d-938b-5d3de2944e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6801","nonce":2284738682}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6802","nonce":2284738682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6804","nonce":2284738682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6803","nonce":2284738682}]},"public_addr":"172.21.15.178:6801/2284738682","cluster_addr":"172.21.15.178:6802/2284738682","heartbeat_back_addr":"172.21.15.178:6804/2284738682","heartbeat_front_addr":"172.21.15.178:6803/2284738682","state":["exists","up"]},{"osd":1,"uuid":"c97cb851-c6dd-4230-ace7-1fe481f29ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6805","nonce":3678228960}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6806","nonce":3678228960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6808","nonce":3678228960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6807","nonce":3678228960}]},"public_addr":"172.21.15.178:6805/3678228960","cluster_addr":"172.21.15.178:6806/3678228960","heartbeat_back_addr":"172.21.15.178:6808/3678228960","heartbeat_front_addr":"172.21.15.178:6807/3678228960","state":["exists","up"]},{"osd":2,"uuid":"bebc9b6f-6349-4f8e-83b1-4177c816c21f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6809","nonce":479756253}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6810","nonce":479756253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6812","nonce":479756253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6811","nonce":479756253}]},"public_addr":"172.21.15.178:6809/479756253","cluster_addr":"172.21.15.178:6810/479756253","heartbeat_back_addr":"172.21.15.178:6812/479756253","heartbeat_front_addr":"172.21.15.178:6811/479756253","state":["exists","up"]},{"osd":3,"uuid":"058ca4e0-7fc0-4b14-93ef-fd9fa2fecd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6813","nonce":731412783}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6814","nonce":731412783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6816","nonce":731412783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6815","nonce":731412783}]},"public_addr":"172.21.15.178:6813/731412783","cluster_addr":"172.21.15.178:6814/731412783","heartbeat_back_addr":"172.21.15.178:6816/731412783","heartbeat_front_addr":"172.21.15.178:6815/731412783","state":["exists","up"]},{"osd":4,"uuid":"c0ea5121-5d8f-48ae-84cd-ba34c8efa7af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6800","nonce":3570504556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6801","nonce":3570504556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6803","nonce":3570504556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6802","nonce":3570504556}]},"public_addr":"172.21.15.184:6800/3570504556","cluster_addr":"172.21.15.184:6801/3570504556","heartbeat_back_addr":"172.21.15.184:6803/3570504556","heartbeat_front_addr":"172.21.15.184:6802/3570504556","state":["exists","up"]},{"osd":5,"uuid":"165cded3-2161-4e1c-93f4-719dd6c7cc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6804","nonce":2863982197}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6805","nonce":2863982197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6807","nonce":2863982197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6806","nonce":2863982197}]},"public_addr":"172.21.15.184:6804/2863982197","cluster_addr":"172.21.15.184:6805/2863982197","heartbeat_back_addr":"172.21.15.184:6807/2863982197","heartbeat_front_addr":"172.21.15.184:6806/2863982197","state":["exists","up"]},{"osd":6,"uuid":"93c78324-6494-44c1-96b1-4c957ae7b33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6808","nonce":2629842997}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6809","nonce":2629842997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6811","nonce":2629842997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6810","nonce":2629842997}]},"public_addr":"172.21.15.184:6808/2629842997","cluster_addr":"172.21.15.184:6809/2629842997","heartbeat_back_addr":"172.21.15.184:6811/2629842997","heartbeat_front_addr":"172.21.15.184:6810/2629842997","state":["exists","up"]},{"osd":7,"uuid":"f4709b8f-a0f5-4623-8b94-db6c8b59f796","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6812","nonce":1589171201}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6813","nonce":1589171201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6815","nonce":1589171201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6814","nonce":1589171201}]},"public_addr":"172.21.15.184:6812/1589171201","cluster_addr":"172.21.15.184:6813/1589171201","heartbeat_back_addr":"172.21.15.184:6815/1589171201","heartbeat_front_addr":"172.21.15.184:6814/1589171201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:38:43.720721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:39:08.896963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:39:34.807815+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:40:01.129025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:40:25.125752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:40:50.590836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:41:16.451927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:41:42.151194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:0/2537478068":"2024-02-15T17:37:20.729807+0000","172.21.15.178:0/373398875":"2024-02-15T17:36:53.862750+0000","172.21.15.178:0/1473396399":"2024-02-15T17:37:20.729807+0000","172.21.15.178:0/2817516403":"2024-02-15T17:36:53.862750+0000","172.21.15.178:6800/1774918016":"2024-02-15T17:36:53.862750+0000","172.21.15.178:0/3369364101":"2024-02-15T17:37:20.729807+0000","172.21.15.178:0/1879459737":"2024-02-15T17:36:53.862750+0000","172.21.15.178:6800/366374299":"2024-02-15T17:37:20.729807+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T17:42:35.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.696+0000 7f0b11951700 1 -- 172.21.15.178:0/3587223187 >> v1:172.21.15.178:6800/1500761607 conn(0x7f0af4073030 legacy=0x7f0af40754f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:42:35.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.696+0000 7f0b11951700 1 -- 172.21.15.178:0/3587223187 >> v1:172.21.15.184:6789/0 conn(0x7f0b0c107a00 legacy=0x7f0b0c10db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:35.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.696+0000 7f0b11951700 1 -- 172.21.15.178:0/3587223187 shutdown_connections 2024-02-14T17:42:35.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.696+0000 7f0b11951700 1 -- 172.21.15.178:0/3587223187 >> 172.21.15.178:0/3587223187 conn(0x7f0b0c0fd530 msgr2=0x7f0b0c1119a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:35.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.696+0000 7f0b11951700 1 -- 172.21.15.178:0/3587223187 shutdown_connections 2024-02-14T17:42:35.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:35.696+0000 7f0b11951700 1 -- 172.21.15.178:0/3587223187 wait complete. 2024-02-14T17:42:35.719 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:35 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:35.621+0000 7fcf2940d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:42:36.073 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3587223187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:42:36.074 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:35.741+0000 7f2dddc7c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:42:36.074 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:35.874+0000 7f2dddc7c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:42:36.074 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3587223187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:42:36.088 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3587223187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:42:36.089 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:35 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:35.752+0000 7fcf2940d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:42:36.333 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:36 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:36.216+0000 7f2dddc7c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:42:36.333 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:36 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:36.308+0000 7f2dddc7c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:42:36.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:36.086+0000 7fcf2940d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:42:36.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:36.176+0000 7fcf2940d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:42:36.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:36.370+0000 7fcf2940d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:42:36.462 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-14T17:42:36.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T17:42:36.716 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:36 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:36.513+0000 7f2dddc7c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:42:36.768 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:36.555+0000 7fcf2940d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:42:36.768 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:36.997 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:36 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:36.714+0000 7f2dddc7c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:42:37.103 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:36.839+0000 7fcf2940d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:42:37.292 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:37 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:37.024+0000 7f2dddc7c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:42:37.293 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:37 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:37.291+0000 7f2dddc7c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:42:37.469 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:37 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:37.100+0000 7fcf2940d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:42:37.470 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:37 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:37.335+0000 7fcf2940d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:42:37.582 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:37 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:37.533+0000 7f2dddc7c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:42:38.469 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:38 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:38.126+0000 7fcf2940d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:42:38.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.588+0000 7ff1936ef700 1 -- 172.21.15.178:0/3430126734 >> v1:172.21.15.178:6789/0 conn(0x7ff18c10a8c0 legacy=0x7ff18c10ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:38.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.588+0000 7ff1936ef700 1 -- 172.21.15.178:0/3430126734 shutdown_connections 2024-02-14T17:42:38.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.588+0000 7ff1936ef700 1 -- 172.21.15.178:0/3430126734 >> 172.21.15.178:0/3430126734 conn(0x7ff18c0fd520 msgr2=0x7ff18c0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:38.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.588+0000 7ff1936ef700 1 -- 172.21.15.178:0/3430126734 shutdown_connections 2024-02-14T17:42:38.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.588+0000 7ff1936ef700 1 -- 172.21.15.178:0/3430126734 wait complete. 2024-02-14T17:42:38.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1936ef700 1 Processor -- start 2024-02-14T17:42:38.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1936ef700 1 -- start start 2024-02-14T17:42:38.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1936ef700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff18c1a8a80 con 0x7ff18c107a10 2024-02-14T17:42:38.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1936ef700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff18c1a9c20 con 0x7ff18c10e490 2024-02-14T17:42:38.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1936ef700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff18c1aadc0 con 0x7ff18c10a8c0 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff19148b700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff18c107a10 0x7ff18c075740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:35946/0 (socket says 172.21.15.178:35946) 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff19148b700 1 -- 172.21.15.178:0/1556130611 learned_addr learned my addr 172.21.15.178:0/1556130611 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114259662 0 0) 0x7ff18c1aadc0 con 0x7ff18c10a8c0 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1780049e0 con 0x7ff18c10a8c0 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027739346 0 0) 0x7ff18c1a8a80 con 0x7ff18c107a10 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff18c1aadc0 con 0x7ff18c107a10 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343885330 0 0) 0x7ff18c1a9c20 con 0x7ff18c10e490 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff18c1a8a80 con 0x7ff18c10e490 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3881315131 0 0) 0x7ff1780049e0 con 0x7ff18c10a8c0 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff18c1a9c20 con 0x7ff18c10a8c0 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.589+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ff17c004320 con 0x7ff18c10a8c0 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3125641556 0 0) 0x7ff18c1a8a80 con 0x7ff18c10e490 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1780049e0 con 0x7ff18c10e490 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ff184001bf0 con 0x7ff18c10e490 2024-02-14T17:42:38.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3984719394 0 0) 0x7ff18c1aadc0 con 0x7ff18c107a10 2024-02-14T17:42:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff18c1a8a80 con 0x7ff18c107a10 2024-02-14T17:42:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ff1880019f0 con 0x7ff18c107a10 2024-02-14T17:42:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.1 v1:172.21.15.184:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2470686339 0 0) 0x7ff1780049e0 con 0x7ff18c10e490 2024-02-14T17:42:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 >> v1:172.21.15.178:6790/0 conn(0x7ff18c10a8c0 legacy=0x7ff18c1a3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 >> v1:172.21.15.178:6789/0 conn(0x7ff18c107a10 legacy=0x7ff18c075740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c1abf60 con 0x7ff18c10e490 2024-02-14T17:42:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1936ef700 1 -- 172.21.15.178:0/1556130611 --> v1:172.21.15.184:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c1aafb0 con 0x7ff18c10e490 2024-02-14T17:42:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1936ef700 1 -- 172.21.15.178:0/1556130611 --> v1:172.21.15.184:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c1ab430 con 0x7ff18c10e490 2024-02-14T17:42:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.1 v1:172.21.15.184:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7ff1840020c0 con 0x7ff18c10e490 2024-02-14T17:42:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.590+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.1 v1:172.21.15.184:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ff184004bb0 con 0x7ff18c10e490 2024-02-14T17:42:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.591+0000 7ff1936ef700 1 -- 172.21.15.178:0/1556130611 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff170005150 con 0x7ff18c10e490 2024-02-14T17:42:38.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.591+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.1 v1:172.21.15.184:6789/0 7 ==== mgrmap(e 17) v1 ==== 91437+0+0 (unknown 2512001928 0 0) 0x7ff18401b330 con 0x7ff18c10e490 2024-02-14T17:42:38.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.592+0000 7ff19148b700 1 -- 172.21.15.178:0/1556130611 >> v1:172.21.15.178:6800/1500761607 conn(0x7ff17806eae0 legacy=0x7ff178070fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1500761607 2024-02-14T17:42:38.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.592+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.1 v1:172.21.15.184:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6001+0+0 (unknown 2253613790 0 0) 0x7ff1840879c0 con 0x7ff18c10e490 2024-02-14T17:42:38.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.596+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.1 v1:172.21.15.184:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7ff184050e70 con 0x7ff18c10e490 2024-02-14T17:42:38.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.793+0000 7ff19148b700 1 -- 172.21.15.178:0/1556130611 >> v1:172.21.15.178:6800/1500761607 conn(0x7ff17806eae0 legacy=0x7ff178070fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.178:6800/1500761607 2024-02-14T17:42:38.795 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:38 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:38.365+0000 7f2dddc7c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:42:38.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.795+0000 7ff1936ef700 1 -- 172.21.15.178:0/1556130611 --> v1:172.21.15.184:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff170005400 con 0x7ff18c10e490 2024-02-14T17:42:38.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.796+0000 7ff1827fc700 1 -- 172.21.15.178:0/1556130611 <== mon.1 v1:172.21.15.184:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+20711 (unknown 2590290900 0 1209335437) 0x7ff184052f70 con 0x7ff18c10e490 2024-02-14T17:42:38.798 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:42:38.799 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":73,"fsid":"6e1588ec-cb5f-11ee-95b9-87774f69a715","created":"2024-02-14T17:36:20.333938+0000","modified":"2024-02-14T17:42:21.816873+0000","last_up_change":"2024-02-14T17:41:44.153032+0000","last_in_change":"2024-02-14T17:41:26.094057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T17:39:39.062242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-14T17:41:55.967442+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-14T17:41:56.498770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-14T17:41:58.450521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-14T17:42:00.289855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-14T17:42:02.364674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"96b590dc-522b-4b8d-938b-5d3de2944e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6801","nonce":2284738682}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6802","nonce":2284738682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6804","nonce":2284738682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6803","nonce":2284738682}]},"public_addr":"172.21.15.178:6801/2284738682","cluster_addr":"172.21.15.178:6802/2284738682","heartbeat_back_addr":"172.21.15.178:6804/2284738682","heartbeat_front_addr":"172.21.15.178:6803/2284738682","state":["exists","up"]},{"osd":1,"uuid":"c97cb851-c6dd-4230-ace7-1fe481f29ae6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6805","nonce":3678228960}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6806","nonce":3678228960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6808","nonce":3678228960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6807","nonce":3678228960}]},"public_addr":"172.21.15.178:6805/3678228960","cluster_addr":"172.21.15.178:6806/3678228960","heartbeat_back_addr":"172.21.15.178:6808/3678228960","heartbeat_front_addr":"172.21.15.178:6807/3678228960","state":["exists","up"]},{"osd":2,"uuid":"bebc9b6f-6349-4f8e-83b1-4177c816c21f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6809","nonce":479756253}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6810","nonce":479756253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6812","nonce":479756253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6811","nonce":479756253}]},"public_addr":"172.21.15.178:6809/479756253","cluster_addr":"172.21.15.178:6810/479756253","heartbeat_back_addr":"172.21.15.178:6812/479756253","heartbeat_front_addr":"172.21.15.178:6811/479756253","state":["exists","up"]},{"osd":3,"uuid":"058ca4e0-7fc0-4b14-93ef-fd9fa2fecd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6813","nonce":731412783}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6814","nonce":731412783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6816","nonce":731412783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.178:6815","nonce":731412783}]},"public_addr":"172.21.15.178:6813/731412783","cluster_addr":"172.21.15.178:6814/731412783","heartbeat_back_addr":"172.21.15.178:6816/731412783","heartbeat_front_addr":"172.21.15.178:6815/731412783","state":["exists","up"]},{"osd":4,"uuid":"c0ea5121-5d8f-48ae-84cd-ba34c8efa7af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6800","nonce":3570504556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6801","nonce":3570504556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6803","nonce":3570504556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6802","nonce":3570504556}]},"public_addr":"172.21.15.184:6800/3570504556","cluster_addr":"172.21.15.184:6801/3570504556","heartbeat_back_addr":"172.21.15.184:6803/3570504556","heartbeat_front_addr":"172.21.15.184:6802/3570504556","state":["exists","up"]},{"osd":5,"uuid":"165cded3-2161-4e1c-93f4-719dd6c7cc0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6804","nonce":2863982197}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6805","nonce":2863982197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6807","nonce":2863982197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6806","nonce":2863982197}]},"public_addr":"172.21.15.184:6804/2863982197","cluster_addr":"172.21.15.184:6805/2863982197","heartbeat_back_addr":"172.21.15.184:6807/2863982197","heartbeat_front_addr":"172.21.15.184:6806/2863982197","state":["exists","up"]},{"osd":6,"uuid":"93c78324-6494-44c1-96b1-4c957ae7b33f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6808","nonce":2629842997}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6809","nonce":2629842997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6811","nonce":2629842997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6810","nonce":2629842997}]},"public_addr":"172.21.15.184:6808/2629842997","cluster_addr":"172.21.15.184:6809/2629842997","heartbeat_back_addr":"172.21.15.184:6811/2629842997","heartbeat_front_addr":"172.21.15.184:6810/2629842997","state":["exists","up"]},{"osd":7,"uuid":"f4709b8f-a0f5-4623-8b94-db6c8b59f796","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6812","nonce":1589171201}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6813","nonce":1589171201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6815","nonce":1589171201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.184:6814","nonce":1589171201}]},"public_addr":"172.21.15.184:6812/1589171201","cluster_addr":"172.21.15.184:6813/1589171201","heartbeat_back_addr":"172.21.15.184:6815/1589171201","heartbeat_front_addr":"172.21.15.184:6814/1589171201","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:38:43.720721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:39:08.896963+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:39:34.807815+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:40:01.129025+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:40:25.125752+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:40:50.590836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:41:16.451927+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:41:42.151194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:0/2537478068":"2024-02-15T17:37:20.729807+0000","172.21.15.178:0/373398875":"2024-02-15T17:36:53.862750+0000","172.21.15.178:0/1473396399":"2024-02-15T17:37:20.729807+0000","172.21.15.178:0/2817516403":"2024-02-15T17:36:53.862750+0000","172.21.15.178:6800/1774918016":"2024-02-15T17:36:53.862750+0000","172.21.15.178:0/3369364101":"2024-02-15T17:37:20.729807+0000","172.21.15.178:0/1879459737":"2024-02-15T17:36:53.862750+0000","172.21.15.178:6800/366374299":"2024-02-15T17:37:20.729807+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T17:42:38.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.799+0000 7ff1936ef700 1 -- 172.21.15.178:0/1556130611 >> v1:172.21.15.178:6800/1500761607 conn(0x7ff17806eae0 legacy=0x7ff178070fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:42:38.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.799+0000 7ff1936ef700 1 -- 172.21.15.178:0/1556130611 >> v1:172.21.15.184:6789/0 conn(0x7ff18c10e490 legacy=0x7ff18c1a7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:38.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.799+0000 7ff1936ef700 1 -- 172.21.15.178:0/1556130611 shutdown_connections 2024-02-14T17:42:38.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.799+0000 7ff1936ef700 1 -- 172.21.15.178:0/1556130611 >> 172.21.15.178:0/1556130611 conn(0x7ff18c0fd520 msgr2=0x7ff18c111120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:38.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.799+0000 7ff1936ef700 1 -- 172.21.15.178:0/1556130611 shutdown_connections 2024-02-14T17:42:38.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:38.799+0000 7ff1936ef700 1 -- 172.21.15.178:0/1556130611 wait complete. 2024-02-14T17:42:38.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1556130611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:42:38.970 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:38 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:38.699+0000 7fcf2940d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:42:38.970 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:38 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:38.797+0000 7fcf2940d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:42:39.065 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1556130611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:42:39.066 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1556130611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:42:39.066 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:38 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:38.962+0000 7f2dddc7c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:42:39.332 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:39.063+0000 7f2dddc7c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:42:39.332 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:39.283+0000 7f2dddc7c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:42:39.436 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:39.014+0000 7fcf2940d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:42:39.436 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: [14/Feb/2024:17:42:39] ENGINE Bus STARTING 2024-02-14T17:42:39.436 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: CherryPy Checker: 2024-02-14T17:42:39.436 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: The Application mounted at '' has an empty config. 2024-02-14T17:42:39.436 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: 2024-02-14T17:42:39.437 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: [14/Feb/2024:17:42:39] ENGINE Serving on http://:::9283 2024-02-14T17:42:39.437 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x[35600]: [14/Feb/2024:17:42:39] ENGINE Bus STARTED 2024-02-14T17:42:39.461 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-14T17:42:39.461 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-14T17:42:39.462 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-14T17:42:39.462 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-14T17:42:39.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-14T17:42:39.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-14T17:42:39.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-14T17:42:39.464 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-14T17:42:39.835 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:42:39] ENGINE Bus STARTING 2024-02-14T17:42:39.836 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: CherryPy Checker: 2024-02-14T17:42:39.836 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: The Application mounted at '' has an empty config. 2024-02-14T17:42:39.836 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T17:42:39.836 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:42:39] ENGINE Serving on http://:::9283 2024-02-14T17:42:39.836 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:42:39] ENGINE Bus STARTED 2024-02-14T17:42:39.999 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:40.002 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:40.002 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:40.168 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: Standby manager daemon x restarted 2024-02-14T17:42:40.168 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: Standby manager daemon x started 2024-02-14T17:42:40.168 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: Active manager daemon y restarted 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: Activating manager daemon y 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: osdmap e74: 8 total, 8 up, 8 in 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: mgrmap e18: y(active, starting, since 0.00816902s), standbys: x 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:42:40.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: Manager daemon y is now available 2024-02-14T17:42:40.170 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: Standby manager daemon x restarted 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: Standby manager daemon x started 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: Active manager daemon y restarted 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: Activating manager daemon y 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: osdmap e74: 8 total, 8 up, 8 in 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: mgrmap e18: y(active, starting, since 0.00816902s), standbys: x 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:40.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: Manager daemon y is now available 2024-02-14T17:42:40.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:40.173 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:40.173 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:40.173 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:42:40.173 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:42:40.173 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:42:40.173 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:42:40.173 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:42:40.173 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:40.173 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:40.174 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:42:40.174 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:42:40.174 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:42:40.174 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:42:40.174 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:39 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:42:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: Standby manager daemon x restarted 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: Standby manager daemon x started 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.? v1:172.21.15.184:0/1488817735' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: Active manager daemon y restarted 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: Activating manager daemon y 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: osdmap e74: 8 total, 8 up, 8 in 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: mgrmap e18: y(active, starting, since 0.00816902s), standbys: x 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:40.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:42:40.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:42:40.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:42:40.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:40.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:40.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:42:40.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:42:40.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:42:40.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:42:40.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:42:40.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:42:40.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:42:40.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:42:40.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: Manager daemon y is now available 2024-02-14T17:42:40.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:40.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:40.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:40.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:42:40.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:42:40.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:42:40.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:42:40.224 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:39 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:42:40.397 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:40.397 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:40.403 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:40.405 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:41.055 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:41.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:40 smithi178 ceph-mon[30321]: [14/Feb/2024:17:42:40] ENGINE Bus STARTING 2024-02-14T17:42:41.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:40 smithi178 ceph-mon[30321]: [14/Feb/2024:17:42:40] ENGINE Serving on https://172.21.15.178:7150 2024-02-14T17:42:41.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:40 smithi178 ceph-mon[30321]: [14/Feb/2024:17:42:40] ENGINE Serving on http://172.21.15.178:8765 2024-02-14T17:42:41.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:40 smithi178 ceph-mon[30321]: [14/Feb/2024:17:42:40] ENGINE Bus STARTED 2024-02-14T17:42:41.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:40 smithi178 ceph-mon[30321]: mgrmap e19: y(active, since 1.01095s), standbys: x 2024-02-14T17:42:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:40 smithi178 ceph-mon[39811]: [14/Feb/2024:17:42:40] ENGINE Bus STARTING 2024-02-14T17:42:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:40 smithi178 ceph-mon[39811]: [14/Feb/2024:17:42:40] ENGINE Serving on https://172.21.15.178:7150 2024-02-14T17:42:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:40 smithi178 ceph-mon[39811]: [14/Feb/2024:17:42:40] ENGINE Serving on http://172.21.15.178:8765 2024-02-14T17:42:41.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:40 smithi178 ceph-mon[39811]: [14/Feb/2024:17:42:40] ENGINE Bus STARTED 2024-02-14T17:42:41.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:40 smithi178 ceph-mon[39811]: mgrmap e19: y(active, since 1.01095s), standbys: x 2024-02-14T17:42:41.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:40 smithi184 ceph-mon[33536]: [14/Feb/2024:17:42:40] ENGINE Bus STARTING 2024-02-14T17:42:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:40 smithi184 ceph-mon[33536]: [14/Feb/2024:17:42:40] ENGINE Serving on https://172.21.15.178:7150 2024-02-14T17:42:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:40 smithi184 ceph-mon[33536]: [14/Feb/2024:17:42:40] ENGINE Serving on http://172.21.15.178:8765 2024-02-14T17:42:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:40 smithi184 ceph-mon[33536]: [14/Feb/2024:17:42:40] ENGINE Bus STARTED 2024-02-14T17:42:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:40 smithi184 ceph-mon[33536]: mgrmap e19: y(active, since 1.01095s), standbys: x 2024-02-14T17:42:42.032 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:41 smithi178 ceph-mon[39811]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:42.032 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:41 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:42.033 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:41 smithi178 ceph-mon[30321]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:42.033 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:41 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:42.165 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:41 smithi184 ceph-mon[33536]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:42.166 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:41 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:43.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:42 smithi178 ceph-mon[39811]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:42 smithi178 ceph-mon[39811]: mgrmap e20: y(active, since 3s), standbys: x 2024-02-14T17:42:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:42 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:42 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:42 smithi178 ceph-mon[30321]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:42 smithi178 ceph-mon[30321]: mgrmap e20: y(active, since 3s), standbys: x 2024-02-14T17:42:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:42 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:42 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:43.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:42 smithi184 ceph-mon[33536]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:43.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:42 smithi184 ceph-mon[33536]: mgrmap e20: y(active, since 3s), standbys: x 2024-02-14T17:42:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:42 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:42 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:44.627 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:44.627 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:44.627 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:44.627 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.627 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.627 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi184 to 3760M 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[39811]: mgrmap e21: y(active, since 4s), standbys: x 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:44.628 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.629 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.629 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.629 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.629 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.629 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.629 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.629 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.629 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi184 to 3760M 2024-02-14T17:42:44.629 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:44.629 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:44 smithi178 ceph-mon[30321]: mgrmap e21: y(active, since 4s), standbys: x 2024-02-14T17:42:44.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:44.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi184 to 3760M 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:42:44.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:44 smithi184 ceph-mon[33536]: mgrmap e21: y(active, since 4s), standbys: x 2024-02-14T17:42:46.059 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:42:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:42:46.687 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:46 smithi178 ceph-mon[39811]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:46.687 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:46 smithi178 ceph-mon[30321]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:46.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.685+0000 7fe74c387700 1 -- 172.21.15.178:0/1371980109 >> v1:172.21.15.178:6789/0 conn(0x7fe74410d380 legacy=0x7fe74410f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:46.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.685+0000 7fe74c387700 1 -- 172.21.15.178:0/1371980109 shutdown_connections 2024-02-14T17:42:46.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.685+0000 7fe74c387700 1 -- 172.21.15.178:0/1371980109 >> 172.21.15.178:0/1371980109 conn(0x7fe7440fc410 msgr2=0x7fe7440fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:46.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.685+0000 7fe74c387700 1 -- 172.21.15.178:0/1371980109 shutdown_connections 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.685+0000 7fe74c387700 1 -- 172.21.15.178:0/1371980109 wait complete. 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.685+0000 7fe74c387700 1 Processor -- start 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe74c387700 1 -- start start 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe74c387700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe7441a6a60 con 0x7fe7441097b0 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe74c387700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe7441a7c40 con 0x7fe74410d380 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe74c387700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe7441a8e40 con 0x7fe744106900 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe749922700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe7441097b0 0x7fe7441a1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50154/0 (socket says 172.21.15.178:50154) 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe749922700 1 -- 172.21.15.178:0/3710094327 learned_addr learned my addr 172.21.15.178:0/3710094327 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590380839 0 0) 0x7fe7441a8e40 con 0x7fe744106900 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7300031f0 con 0x7fe744106900 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44776244 0 0) 0x7fe7441a7c40 con 0x7fe74410d380 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7441a8e40 con 0x7fe74410d380 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608737071 0 0) 0x7fe7441a6a60 con 0x7fe7441097b0 2024-02-14T17:42:46.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7441a7c40 con 0x7fe7441097b0 2024-02-14T17:42:46.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3878639061 0 0) 0x7fe7300031f0 con 0x7fe744106900 2024-02-14T17:42:46.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe7441a6a60 con 0x7fe744106900 2024-02-14T17:42:46.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.686+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fe7340018d0 con 0x7fe744106900 2024-02-14T17:42:46.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3498967345 0 0) 0x7fe7441a8e40 con 0x7fe74410d380 2024-02-14T17:42:46.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe7300031f0 con 0x7fe74410d380 2024-02-14T17:42:46.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fe73c0017b0 con 0x7fe74410d380 2024-02-14T17:42:46.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1482021680 0 0) 0x7fe7441a7c40 con 0x7fe7441097b0 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe7441a8e40 con 0x7fe7441097b0 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3117428289 0 0) 0x7fe7441a6a60 con 0x7fe744106900 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 >> v1:172.21.15.184:6789/0 conn(0x7fe74410d380 legacy=0x7fe7441a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 >> v1:172.21.15.178:6789/0 conn(0x7fe7441097b0 legacy=0x7fe7441a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7441aa040 con 0x7fe744106900 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe74c387700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7441a7e30 con 0x7fe744106900 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe74c387700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7441a82b0 con 0x7fe744106900 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fe734003b10 con 0x7fe744106900 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.687+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fe734004930 con 0x7fe744106900 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.688+0000 7fe74c387700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.178:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe728000fb0 con 0x7fe744106900 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.688+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7fe73401b0a0 con 0x7fe744106900 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.689+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7fe7340876d0 con 0x7fe744106900 2024-02-14T17:42:46.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.689+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.178:6801/2284738682 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe73007cfc0 con 0x7fe7300792b0 2024-02-14T17:42:46.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.689+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (unknown 3287965567 0 0) 0x7fe734087a20 con 0x7fe744106900 2024-02-14T17:42:46.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.690+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== osd.0 v1:172.21.15.178:6801/2284738682 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fe73007cfc0 con 0x7fe7300792b0 2024-02-14T17:42:46.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.706+0000 7fe74c387700 1 -- 172.21.15.178:0/3710094327 --> v1:172.21.15.178:6801/2284738682 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe728002c70 con 0x7fe7300792b0 2024-02-14T17:42:46.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.707+0000 7fe73b7fe700 1 -- 172.21.15.178:0/3710094327 <== osd.0 v1:172.21.15.178:6801/2284738682 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1002017476) 0x7fe728002c70 con 0x7fe7300792b0 2024-02-14T17:42:46.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.707+0000 7fe74c387700 1 -- 172.21.15.178:0/3710094327 >> v1:172.21.15.178:6801/2284738682 conn(0x7fe7300792b0 legacy=0x7fe73007b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:46.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.707+0000 7fe74c387700 1 -- 172.21.15.178:0/3710094327 >> v1:172.21.15.178:6800/4205070280 conn(0x7fe73006ebd0 legacy=0x7fe730071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:46.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.707+0000 7fe74c387700 1 -- 172.21.15.178:0/3710094327 >> v1:172.21.15.178:6790/0 conn(0x7fe744106900 legacy=0x7fe74410cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:46.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.707+0000 7fe74a924700 1 -- 172.21.15.178:0/3710094327 reap_dead start 2024-02-14T17:42:46.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.707+0000 7fe74c387700 1 -- 172.21.15.178:0/3710094327 shutdown_connections 2024-02-14T17:42:46.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.707+0000 7fe74c387700 1 -- 172.21.15.178:0/3710094327 >> 172.21.15.178:0/3710094327 conn(0x7fe7440fc410 msgr2=0x7fe7441101f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:46.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.707+0000 7fe74c387700 1 -- 172.21.15.178:0/3710094327 shutdown_connections 2024-02-14T17:42:46.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:46.707+0000 7fe74c387700 1 -- 172.21.15.178:0/3710094327 wait complete. 2024-02-14T17:42:46.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:46 smithi184 ceph-mon[33536]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:42:48.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.671+0000 7f1fe119f700 1 -- 172.21.15.178:0/1859645884 >> v1:172.21.15.178:6790/0 conn(0x7f1fdc10d380 legacy=0x7f1fdc10f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:48.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.671+0000 7f1fe119f700 1 -- 172.21.15.178:0/1859645884 shutdown_connections 2024-02-14T17:42:48.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.671+0000 7f1fe119f700 1 -- 172.21.15.178:0/1859645884 >> 172.21.15.178:0/1859645884 conn(0x7f1fdc0fc410 msgr2=0x7f1fdc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:48.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.671+0000 7f1fe119f700 1 -- 172.21.15.178:0/1859645884 shutdown_connections 2024-02-14T17:42:48.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.671+0000 7f1fe119f700 1 -- 172.21.15.178:0/1859645884 wait complete. 2024-02-14T17:42:48.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fe119f700 1 Processor -- start 2024-02-14T17:42:48.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fe119f700 1 -- start start 2024-02-14T17:42:48.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fe119f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1fdc1a6910 con 0x7f1fdc1097b0 2024-02-14T17:42:48.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fe119f700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1fdc1a7af0 con 0x7f1fdc106900 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fe119f700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1fdc1a8cf0 con 0x7f1fdc10d380 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fda59c700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f1fdc1097b0 0x7f1fdc1a1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50176/0 (socket says 172.21.15.178:50176) 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fdb59e700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f1fdc10d380 0x7f1fdc1a4fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:37656/0 (socket says 172.21.15.178:37656) 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fda59c700 1 -- 172.21.15.178:0/175388039 learned_addr learned my addr 172.21.15.178:0/175388039 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475010728 0 0) 0x7f1fdc1a8cf0 con 0x7f1fdc10d380 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc40035b0 con 0x7f1fdc10d380 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674691345 0 0) 0x7f1fdc1a6910 con 0x7f1fdc1097b0 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.672+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fdc1a8cf0 con 0x7f1fdc1097b0 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200713935 0 0) 0x7f1fdc1a7af0 con 0x7f1fdc106900 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fdc1a6910 con 0x7f1fdc106900 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1826639131 0 0) 0x7f1fdc1a8cf0 con 0x7f1fdc1097b0 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1fdc1a7af0 con 0x7f1fdc1097b0 2024-02-14T17:42:48.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f1fcc0018f0 con 0x7f1fdc1097b0 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2971841236 0 0) 0x7f1fc40035b0 con 0x7f1fdc10d380 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1fdc1a8cf0 con 0x7f1fdc10d380 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f1fd4004630 con 0x7f1fdc10d380 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1693836106 0 0) 0x7f1fdc1a7af0 con 0x7f1fdc1097b0 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 >> v1:172.21.15.178:6790/0 conn(0x7f1fdc10d380 legacy=0x7f1fdc1a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 >> v1:172.21.15.184:6789/0 conn(0x7f1fdc106900 legacy=0x7f1fdc10cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc1a9ef0 con 0x7f1fdc1097b0 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fe119f700 1 -- 172.21.15.178:0/175388039 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fdc1a6b00 con 0x7f1fdc1097b0 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fe119f700 1 -- 172.21.15.178:0/175388039 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fdc1a6f80 con 0x7f1fdc1097b0 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f1fcc001c80 con 0x7f1fdc1097b0 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.673+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f1fcc005cc0 con 0x7f1fdc1097b0 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.674+0000 7f1fe119f700 1 -- 172.21.15.178:0/175388039 --> v1:172.21.15.178:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1fc8000f80 con 0x7f1fdc1097b0 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.674+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f1fcc01c350 con 0x7f1fdc1097b0 2024-02-14T17:42:48.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.675+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7f1fcc088a20 con 0x7f1fdc1097b0 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.675+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 --> v1:172.21.15.184:6800/3570504556 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1fc40810b0 con 0x7f1fc407d3a0 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.675+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (unknown 3287965567 0 0) 0x7f1fcc088d90 con 0x7f1fdc1097b0 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.677+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== osd.4 v1:172.21.15.184:6800/3570504556 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f1fc40810b0 con 0x7f1fc407d3a0 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.695+0000 7f1fe119f700 1 -- 172.21.15.178:0/175388039 --> v1:172.21.15.184:6800/3570504556 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1fc8002c40 con 0x7f1fc407d3a0 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.696+0000 7f1fc3fff700 1 -- 172.21.15.178:0/175388039 <== osd.4 v1:172.21.15.184:6800/3570504556 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3087939264) 0x7f1fc8002c40 con 0x7f1fc407d3a0 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.696+0000 7f1fe119f700 1 -- 172.21.15.178:0/175388039 >> v1:172.21.15.184:6800/3570504556 conn(0x7f1fc407d3a0 legacy=0x7f1fc407f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.696+0000 7f1fe119f700 1 -- 172.21.15.178:0/175388039 >> v1:172.21.15.178:6800/4205070280 conn(0x7f1fc4072cc0 legacy=0x7f1fc4075180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.696+0000 7f1fe119f700 1 -- 172.21.15.178:0/175388039 >> v1:172.21.15.178:6789/0 conn(0x7f1fdc1097b0 legacy=0x7f1fdc1a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.696+0000 7f1fdb59e700 1 -- 172.21.15.178:0/175388039 reap_dead start 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.696+0000 7f1fe119f700 1 -- 172.21.15.178:0/175388039 shutdown_connections 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.696+0000 7f1fe119f700 1 -- 172.21.15.178:0/175388039 >> 172.21.15.178:0/175388039 conn(0x7f1fdc0fc410 msgr2=0x7f1fdc100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.696+0000 7f1fe119f700 1 -- 172.21.15.178:0/175388039 shutdown_connections 2024-02-14T17:42:48.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.696+0000 7f1fe119f700 1 -- 172.21.15.178:0/175388039 wait complete. 2024-02-14T17:42:48.821 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:48 smithi178 ceph-mon[39811]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-14T17:42:48.821 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:48 smithi178 ceph-mon[30321]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-14T17:42:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:48 smithi184 ceph-mon[33536]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-14T17:42:49.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.786+0000 7f4090ad6700 1 -- 172.21.15.178:0/7521271 >> v1:172.21.15.178:6789/0 conn(0x7f408c1097c0 legacy=0x7f408c10bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:49.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.786+0000 7f4090ad6700 1 -- 172.21.15.178:0/7521271 shutdown_connections 2024-02-14T17:42:49.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.786+0000 7f4090ad6700 1 -- 172.21.15.178:0/7521271 >> 172.21.15.178:0/7521271 conn(0x7f408c0fc420 msgr2=0x7f408c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:49.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.786+0000 7f4090ad6700 1 -- 172.21.15.178:0/7521271 shutdown_connections 2024-02-14T17:42:49.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.786+0000 7f4090ad6700 1 -- 172.21.15.178:0/7521271 wait complete. 2024-02-14T17:42:49.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.787+0000 7f4090ad6700 1 Processor -- start 2024-02-14T17:42:49.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.787+0000 7f4090ad6700 1 -- start start 2024-02-14T17:42:49.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.787+0000 7f4090ad6700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f408c1a5790 con 0x7f408c106910 2024-02-14T17:42:49.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.787+0000 7f4090ad6700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f408c1a6970 con 0x7f408c10d390 2024-02-14T17:42:49.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.787+0000 7f4090ad6700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f408c1a7b70 con 0x7f408c1097c0 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.787+0000 7f4089d9b700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f408c1097c0 0x7f408c1a0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:37684/0 (socket says 172.21.15.178:37684) 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f4089d9b700 1 -- 172.21.15.178:0/1806501776 learned_addr learned my addr 172.21.15.178:0/1806501776 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149568978 0 0) 0x7f408c1a7b70 con 0x7f408c1097c0 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40740031f0 con 0x7f408c1097c0 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505874669 0 0) 0x7f408c1a5790 con 0x7f408c106910 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f408c1a7b70 con 0x7f408c106910 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024630549 0 0) 0x7f408c1a6970 con 0x7f408c10d390 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f408c1a5790 con 0x7f408c10d390 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3158790332 0 0) 0x7f408c1a7b70 con 0x7f408c106910 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f408c1a6970 con 0x7f408c106910 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f407c001830 con 0x7f408c106910 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3776570162 0 0) 0x7f40740031f0 con 0x7f408c1097c0 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f408c1a7b70 con 0x7f408c1097c0 2024-02-14T17:42:49.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f4080004540 con 0x7f408c1097c0 2024-02-14T17:42:49.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2601573788 0 0) 0x7f408c1a5790 con 0x7f408c10d390 2024-02-14T17:42:49.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f40740031f0 con 0x7f408c10d390 2024-02-14T17:42:49.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f4084001b00 con 0x7f408c10d390 2024-02-14T17:42:49.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4061017467 0 0) 0x7f408c1a6970 con 0x7f408c106910 2024-02-14T17:42:49.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 >> v1:172.21.15.178:6790/0 conn(0x7f408c1097c0 legacy=0x7f408c1a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:49.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 >> v1:172.21.15.184:6789/0 conn(0x7f408c10d390 legacy=0x7f408c1a3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:49.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c1a8d70 con 0x7f408c106910 2024-02-14T17:42:49.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f4090ad6700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408c1a5980 con 0x7f408c106910 2024-02-14T17:42:49.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f4090ad6700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f408c1a5e00 con 0x7f408c106910 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f407c003b30 con 0x7f408c106910 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.788+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f407c004a60 con 0x7f408c106910 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.789+0000 7f4090ad6700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.178:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f406c000f80 con 0x7f408c106910 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.790+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f407c0036a0 con 0x7f408c106910 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.790+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7f407c087610 con 0x7f408c106910 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.790+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.184:6804/2863982197 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f407407cdf0 con 0x7f40740790e0 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.790+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (unknown 3287965567 0 0) 0x7f407c087aa0 con 0x7f408c106910 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.792+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== osd.5 v1:172.21.15.184:6804/2863982197 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f407407cdf0 con 0x7f40740790e0 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.808+0000 7f4090ad6700 1 -- 172.21.15.178:0/1806501776 --> v1:172.21.15.184:6804/2863982197 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f406c002c40 con 0x7f40740790e0 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.808+0000 7f407b7fe700 1 -- 172.21.15.178:0/1806501776 <== osd.5 v1:172.21.15.184:6804/2863982197 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 398800033) 0x7f406c002c40 con 0x7f40740790e0 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.809+0000 7f4090ad6700 1 -- 172.21.15.178:0/1806501776 >> v1:172.21.15.184:6804/2863982197 conn(0x7f40740790e0 legacy=0x7f407407b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.809+0000 7f4090ad6700 1 -- 172.21.15.178:0/1806501776 >> v1:172.21.15.178:6800/4205070280 conn(0x7f407406ea00 legacy=0x7f4074070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.809+0000 7f4090ad6700 1 -- 172.21.15.178:0/1806501776 >> v1:172.21.15.178:6789/0 conn(0x7f408c106910 legacy=0x7f408c10c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.809+0000 7f408ad9d700 1 -- 172.21.15.178:0/1806501776 reap_dead start 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.809+0000 7f4090ad6700 1 -- 172.21.15.178:0/1806501776 shutdown_connections 2024-02-14T17:42:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.809+0000 7f4090ad6700 1 -- 172.21.15.178:0/1806501776 >> 172.21.15.178:0/1806501776 conn(0x7f408c0fc420 msgr2=0x7f408c0feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:49.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.809+0000 7f4090ad6700 1 -- 172.21.15.178:0/1806501776 shutdown_connections 2024-02-14T17:42:49.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:48.809+0000 7f4090ad6700 1 -- 172.21.15.178:0/1806501776 wait complete. 2024-02-14T17:42:50.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:50 smithi178 ceph-mon[39811]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-14T17:42:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:50 smithi178 ceph-mon[30321]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-14T17:42:50.852 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:50 smithi184 ceph-mon[33536]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-14T17:42:50.882 INFO:teuthology.orchestra.run.smithi178.stdout:133143986206 2024-02-14T17:42:50.883 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-14T17:42:51.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:42:50 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:42:51.828 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:51 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:51.828 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:51 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:51.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:51 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:42:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.116+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3004442313 >> v1:172.21.15.178:6789/0 conn(0x7f5e9c106900 legacy=0x7f5e9c106ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.116+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3004442313 shutdown_connections 2024-02-14T17:42:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.116+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3004442313 >> 172.21.15.178:0/3004442313 conn(0x7f5e9c0fc410 msgr2=0x7f5e9c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.117+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3004442313 shutdown_connections 2024-02-14T17:42:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.117+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3004442313 wait complete. 2024-02-14T17:42:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.117+0000 7f5ea10d5700 1 Processor -- start 2024-02-14T17:42:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5ea10d5700 1 -- start start 2024-02-14T17:42:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5ea10d5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5e9c1a5880 con 0x7f5e9c10d380 2024-02-14T17:42:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5ea10d5700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5e9c1a6a60 con 0x7f5e9c106900 2024-02-14T17:42:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5ea10d5700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5e9c1a7c60 con 0x7f5e9c1097b0 2024-02-14T17:42:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5e9a59c700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f5e9c1097b0 0x7f5e9c1a0730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:37708/0 (socket says 172.21.15.178:37708) 2024-02-14T17:42:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5e9a59c700 1 -- 172.21.15.178:0/3428835678 learned_addr learned my addr 172.21.15.178:0/3428835678 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142526292 0 0) 0x7f5e9c1a7c60 con 0x7f5e9c1097b0 2024-02-14T17:42:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e840035b0 con 0x7f5e9c1097b0 2024-02-14T17:42:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668823783 0 0) 0x7f5e9c1a5880 con 0x7f5e9c10d380 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e9c1a7c60 con 0x7f5e9c10d380 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340872781 0 0) 0x7f5e9c1a6a60 con 0x7f5e9c106900 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e9c1a5880 con 0x7f5e9c106900 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.118+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2259830462 0 0) 0x7f5e840035b0 con 0x7f5e9c1097b0 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e9c1a6a60 con 0x7f5e9c1097b0 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3386760287 0 0) 0x7f5e9c1a7c60 con 0x7f5e9c10d380 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e840035b0 con 0x7f5e9c10d380 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f5e900018d0 con 0x7f5e9c1097b0 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f5e94001af0 con 0x7f5e9c10d380 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2014171592 0 0) 0x7f5e9c1a5880 con 0x7f5e9c106900 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e9c1a7c60 con 0x7f5e9c106900 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3367822674 0 0) 0x7f5e9c1a6a60 con 0x7f5e9c1097b0 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 >> v1:172.21.15.184:6789/0 conn(0x7f5e9c106900 legacy=0x7f5e9c10ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 >> v1:172.21.15.178:6789/0 conn(0x7f5e9c10d380 legacy=0x7f5e9c1a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c1a8e60 con 0x7f5e9c1097b0 2024-02-14T17:42:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9c1a7e50 con 0x7f5e9c1097b0 2024-02-14T17:42:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9c1a82d0 con 0x7f5e9c1097b0 2024-02-14T17:42:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f5e90001a80 con 0x7f5e9c1097b0 2024-02-14T17:42:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.119+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f5e90005b90 con 0x7f5e9c1097b0 2024-02-14T17:42:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.120+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.178:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e88000fb0 con 0x7f5e9c1097b0 2024-02-14T17:42:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.120+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f5e90003680 con 0x7f5e9c1097b0 2024-02-14T17:42:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.121+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7f5e90088850 con 0x7f5e9c1097b0 2024-02-14T17:42:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.121+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.178:6805/3678228960 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e8407d280 con 0x7f5e84079570 2024-02-14T17:42:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.121+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (unknown 3287965567 0 0) 0x7f5e90088c20 con 0x7f5e9c1097b0 2024-02-14T17:42:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.122+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== osd.1 v1:172.21.15.178:6805/3678228960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f5e8407d280 con 0x7f5e84079570 2024-02-14T17:42:52.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.139+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3428835678 --> v1:172.21.15.178:6805/3678228960 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5e88002c70 con 0x7f5e84079570 2024-02-14T17:42:52.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.140+0000 7f5e83fff700 1 -- 172.21.15.178:0/3428835678 <== osd.1 v1:172.21.15.178:6805/3678228960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 781486577) 0x7f5e88002c70 con 0x7f5e84079570 2024-02-14T17:42:52.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.140+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3428835678 >> v1:172.21.15.178:6805/3678228960 conn(0x7f5e84079570 legacy=0x7f5e8407ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:52.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.140+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3428835678 >> v1:172.21.15.178:6800/4205070280 conn(0x7f5e8406ecb0 legacy=0x7f5e84071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:52.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.140+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3428835678 >> v1:172.21.15.178:6790/0 conn(0x7f5e9c1097b0 legacy=0x7f5e9c1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:52.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.140+0000 7f5e9b59e700 1 -- 172.21.15.178:0/3428835678 reap_dead start 2024-02-14T17:42:52.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.140+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3428835678 shutdown_connections 2024-02-14T17:42:52.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.140+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3428835678 >> 172.21.15.178:0/3428835678 conn(0x7f5e9c0fc410 msgr2=0x7f5e9c100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:52.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.140+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3428835678 shutdown_connections 2024-02-14T17:42:52.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:52.140+0000 7f5ea10d5700 1 -- 172.21.15.178:0/3428835678 wait complete. 2024-02-14T17:42:52.812 INFO:teuthology.orchestra.run.smithi178.stdout:34359738418 2024-02-14T17:42:52.812 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-14T17:42:52.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:52 smithi178 ceph-mon[39811]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-14T17:42:52.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:52 smithi178 ceph-mon[30321]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-14T17:42:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:52 smithi184 ceph-mon[33536]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-14T17:42:53.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.272+0000 7fc7a08a5700 1 -- 172.21.15.178:0/3444973288 >> v1:172.21.15.178:6789/0 conn(0x7fc79c10a8e0 legacy=0x7fc79c10ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:53.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.272+0000 7fc7a08a5700 1 -- 172.21.15.178:0/3444973288 shutdown_connections 2024-02-14T17:42:53.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.272+0000 7fc7a08a5700 1 -- 172.21.15.178:0/3444973288 >> 172.21.15.178:0/3444973288 conn(0x7fc79c0fd560 msgr2=0x7fc79c0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:53.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.273+0000 7fc7a08a5700 1 -- 172.21.15.178:0/3444973288 shutdown_connections 2024-02-14T17:42:53.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.273+0000 7fc7a08a5700 1 -- 172.21.15.178:0/3444973288 wait complete. 2024-02-14T17:42:53.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.273+0000 7fc7a08a5700 1 Processor -- start 2024-02-14T17:42:53.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.273+0000 7fc7a08a5700 1 -- start start 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc7a08a5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc79c1a8b30 con 0x7fc79c10e4b0 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc7a08a5700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc79c1a9cd0 con 0x7fc79c107a30 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc7a08a5700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc79c1aaeb0 con 0x7fc79c10a8e0 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc79ad9d700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc79c10e4b0 0x7fc79c1a7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50244/0 (socket says 172.21.15.178:50244) 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc79ad9d700 1 -- 172.21.15.178:0/1602103554 learned_addr learned my addr 172.21.15.178:0/1602103554 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc799d9b700 1 --1- 172.21.15.178:0/1602103554 >> v1:172.21.15.178:6790/0 conn(0x7fc79c10a8e0 0x7fc79c1a3c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:37724/0 (socket says 172.21.15.178:37724) 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230753395 0 0) 0x7fc79c1a8b30 con 0x7fc79c10e4b0 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7840049e0 con 0x7fc79c10e4b0 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99058914 0 0) 0x7fc79c1aaeb0 con 0x7fc79c10a8e0 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79c1a8b30 con 0x7fc79c10a8e0 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3903068566 0 0) 0x7fc79c1a8b30 con 0x7fc79c10a8e0 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc79c1aaeb0 con 0x7fc79c10a8e0 2024-02-14T17:42:53.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fc790004320 con 0x7fc79c10a8e0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1812726304 0 0) 0x7fc7840049e0 con 0x7fc79c10e4b0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc79c1a8b30 con 0x7fc79c10e4b0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fc794001c50 con 0x7fc79c10e4b0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871722024 0 0) 0x7fc79c1a9cd0 con 0x7fc79c107a30 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7840049e0 con 0x7fc79c107a30 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 702892196 0 0) 0x7fc79c1aaeb0 con 0x7fc79c10a8e0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 >> v1:172.21.15.184:6789/0 conn(0x7fc79c107a30 legacy=0x7fc79c075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 >> v1:172.21.15.178:6789/0 conn(0x7fc79c10e4b0 legacy=0x7fc79c1a7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c1ac090 con 0x7fc79c10a8e0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc7a08a5700 1 -- 172.21.15.178:0/1602103554 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79c1ab0a0 con 0x7fc79c10a8e0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.274+0000 7fc7a08a5700 1 -- 172.21.15.178:0/1602103554 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79c1ab520 con 0x7fc79c10a8e0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.275+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fc790003ba0 con 0x7fc79c10a8e0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.275+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fc790004ed0 con 0x7fc79c10a8e0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.275+0000 7fc7a08a5700 1 -- 172.21.15.178:0/1602103554 --> v1:172.21.15.178:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc77c000fb0 con 0x7fc79c10a8e0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.276+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7fc790005050 con 0x7fc79c10a8e0 2024-02-14T17:42:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.276+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7fc790088be0 con 0x7fc79c10a8e0 2024-02-14T17:42:53.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.276+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 --> v1:172.21.15.184:6812/1589171201 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc78407cf60 con 0x7fc784079250 2024-02-14T17:42:53.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.276+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (unknown 3287965567 0 0) 0x7fc790089010 con 0x7fc79c10a8e0 2024-02-14T17:42:53.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.278+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== osd.7 v1:172.21.15.184:6812/1589171201 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fc78407cf60 con 0x7fc784079250 2024-02-14T17:42:53.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.294+0000 7fc7a08a5700 1 -- 172.21.15.178:0/1602103554 --> v1:172.21.15.184:6812/1589171201 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc77c002c70 con 0x7fc784079250 2024-02-14T17:42:53.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.295+0000 7fc78b7fe700 1 -- 172.21.15.178:0/1602103554 <== osd.7 v1:172.21.15.184:6812/1589171201 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 603386595) 0x7fc77c002c70 con 0x7fc784079250 2024-02-14T17:42:53.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.295+0000 7fc7a08a5700 1 -- 172.21.15.178:0/1602103554 >> v1:172.21.15.184:6812/1589171201 conn(0x7fc784079250 legacy=0x7fc78407b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:53.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.295+0000 7fc7a08a5700 1 -- 172.21.15.178:0/1602103554 >> v1:172.21.15.178:6800/4205070280 conn(0x7fc78406ea70 legacy=0x7fc784070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:53.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.295+0000 7fc7a08a5700 1 -- 172.21.15.178:0/1602103554 >> v1:172.21.15.178:6790/0 conn(0x7fc79c10a8e0 legacy=0x7fc79c1a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:53.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.295+0000 7fc79ad9d700 1 -- 172.21.15.178:0/1602103554 reap_dead start 2024-02-14T17:42:53.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.295+0000 7fc7a08a5700 1 -- 172.21.15.178:0/1602103554 shutdown_connections 2024-02-14T17:42:53.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.295+0000 7fc7a08a5700 1 -- 172.21.15.178:0/1602103554 >> 172.21.15.178:0/1602103554 conn(0x7fc79c0fd560 msgr2=0x7fc79c1119d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:53.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.295+0000 7fc7a08a5700 1 -- 172.21.15.178:0/1602103554 shutdown_connections 2024-02-14T17:42:53.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:53.295+0000 7fc7a08a5700 1 -- 172.21.15.178:0/1602103554 wait complete. 2024-02-14T17:42:53.735 INFO:teuthology.orchestra.run.smithi178.stdout:167503724568 2024-02-14T17:42:53.735 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-14T17:42:54.344 INFO:teuthology.orchestra.run.smithi178.stdout:240518168592 2024-02-14T17:42:54.344 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-14T17:42:54.905 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:54 smithi178 ceph-mon[39811]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:42:54.905 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:54 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:42:54.906 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:54 smithi178 ceph-mon[30321]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:42:54.906 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:54 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:42:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:54 smithi184 ceph-mon[33536]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:42:54.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:54 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:42:54.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.906+0000 7f85f4f43700 1 -- 172.21.15.178:0/916043031 >> v1:172.21.15.178:6789/0 conn(0x7f85f0109860 legacy=0x7f85f010bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:54.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.907+0000 7f85f4f43700 1 -- 172.21.15.178:0/916043031 shutdown_connections 2024-02-14T17:42:54.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.907+0000 7f85f4f43700 1 -- 172.21.15.178:0/916043031 >> 172.21.15.178:0/916043031 conn(0x7f85f0075c10 msgr2=0x7f85f0076020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.907+0000 7f85f4f43700 1 -- 172.21.15.178:0/916043031 shutdown_connections 2024-02-14T17:42:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.907+0000 7f85f4f43700 1 -- 172.21.15.178:0/916043031 wait complete. 2024-02-14T17:42:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.907+0000 7f85f4f43700 1 Processor -- start 2024-02-14T17:42:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.907+0000 7f85f4f43700 1 -- start start 2024-02-14T17:42:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85f4f43700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f85f01a58f0 con 0x7f85f010d400 2024-02-14T17:42:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85f4f43700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f85f01a6ad0 con 0x7f85f0109860 2024-02-14T17:42:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85f4f43700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f85f01a7cd0 con 0x7f85f0101600 2024-02-14T17:42:54.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85eed9d700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f85f010d400 0x7f85f01a3fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:34286/0 (socket says 172.21.15.178:34286) 2024-02-14T17:42:54.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85ee59c700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f85f0101600 0x7f85f0105e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:34462/0 (socket says 172.21.15.178:34462) 2024-02-14T17:42:54.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85eed9d700 1 -- 172.21.15.178:0/2254080192 learned_addr learned my addr 172.21.15.178:0/2254080192 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:54.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205365948 0 0) 0x7f85f01a58f0 con 0x7f85f010d400 2024-02-14T17:42:54.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85d80035b0 con 0x7f85f010d400 2024-02-14T17:42:54.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811498373 0 0) 0x7f85f01a7cd0 con 0x7f85f0101600 2024-02-14T17:42:54.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85f01a58f0 con 0x7f85f0101600 2024-02-14T17:42:54.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260642629 0 0) 0x7f85f01a6ad0 con 0x7f85f0109860 2024-02-14T17:42:54.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85f01a7cd0 con 0x7f85f0109860 2024-02-14T17:42:54.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 864099925 0 0) 0x7f85d80035b0 con 0x7f85f010d400 2024-02-14T17:42:54.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f85f01a6ad0 con 0x7f85f010d400 2024-02-14T17:42:54.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.908+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f85e8001bd0 con 0x7f85f010d400 2024-02-14T17:42:54.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 558422426 0 0) 0x7f85f01a58f0 con 0x7f85f0101600 2024-02-14T17:42:54.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f85d80035b0 con 0x7f85f0101600 2024-02-14T17:42:54.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3729939544 0 0) 0x7f85f01a7cd0 con 0x7f85f0109860 2024-02-14T17:42:54.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f85f01a58f0 con 0x7f85f0109860 2024-02-14T17:42:54.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f85e0001770 con 0x7f85f0101600 2024-02-14T17:42:54.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f85e4003820 con 0x7f85f0109860 2024-02-14T17:42:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 359045063 0 0) 0x7f85f01a6ad0 con 0x7f85f010d400 2024-02-14T17:42:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 >> v1:172.21.15.178:6790/0 conn(0x7f85f0101600 legacy=0x7f85f0105e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 >> v1:172.21.15.184:6789/0 conn(0x7f85f0109860 legacy=0x7f85f01a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f01a8ed0 con 0x7f85f010d400 2024-02-14T17:42:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85f4f43700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f01a7ec0 con 0x7f85f010d400 2024-02-14T17:42:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85f4f43700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f01a8340 con 0x7f85f010d400 2024-02-14T17:42:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f85e8003eb0 con 0x7f85f010d400 2024-02-14T17:42:54.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.909+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f85e8005ee0 con 0x7f85f010d400 2024-02-14T17:42:54.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.910+0000 7f85f4f43700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.178:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85d0000f80 con 0x7f85f010d400 2024-02-14T17:42:54.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.910+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f85e801c570 con 0x7f85f010d400 2024-02-14T17:42:54.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.911+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7f85e8088cf0 con 0x7f85f010d400 2024-02-14T17:42:54.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.911+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.178:6813/731412783 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85d807d160 con 0x7f85d8079450 2024-02-14T17:42:54.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.911+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (unknown 3287965567 0 0) 0x7f85e8089100 con 0x7f85f010d400 2024-02-14T17:42:54.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.912+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== osd.3 v1:172.21.15.178:6813/731412783 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f85d807d160 con 0x7f85d8079450 2024-02-14T17:42:54.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.929+0000 7f85f4f43700 1 -- 172.21.15.178:0/2254080192 --> v1:172.21.15.178:6813/731412783 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f85d0002c40 con 0x7f85d8079450 2024-02-14T17:42:54.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.929+0000 7f85df7fe700 1 -- 172.21.15.178:0/2254080192 <== osd.3 v1:172.21.15.178:6813/731412783 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1221308824) 0x7f85d0002c40 con 0x7f85d8079450 2024-02-14T17:42:54.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.929+0000 7f85f4f43700 1 -- 172.21.15.178:0/2254080192 >> v1:172.21.15.178:6813/731412783 conn(0x7f85d8079450 legacy=0x7f85d807b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:54.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.929+0000 7f85f4f43700 1 -- 172.21.15.178:0/2254080192 >> v1:172.21.15.178:6800/4205070280 conn(0x7f85d806ebf0 legacy=0x7f85d80710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:54.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.929+0000 7f85f4f43700 1 -- 172.21.15.178:0/2254080192 >> v1:172.21.15.178:6789/0 conn(0x7f85f010d400 legacy=0x7f85f01a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:54.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.930+0000 7f85eed9d700 1 -- 172.21.15.178:0/2254080192 reap_dead start 2024-02-14T17:42:54.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.930+0000 7f85f4f43700 1 -- 172.21.15.178:0/2254080192 shutdown_connections 2024-02-14T17:42:54.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.930+0000 7f85f4f43700 1 -- 172.21.15.178:0/2254080192 >> 172.21.15.178:0/2254080192 conn(0x7f85f0075c10 msgr2=0x7f85f010f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:54.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.930+0000 7f85f4f43700 1 -- 172.21.15.178:0/2254080192 shutdown_connections 2024-02-14T17:42:54.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:54.930+0000 7f85f4f43700 1 -- 172.21.15.178:0/2254080192 wait complete. 2024-02-14T17:42:55.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.362+0000 7fdddecd9700 1 -- 172.21.15.178:0/1937121841 >> v1:172.21.15.178:6790/0 conn(0x7fddd8114ab0 legacy=0x7fddd8116f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:55.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.362+0000 7fdddecd9700 1 -- 172.21.15.178:0/1937121841 shutdown_connections 2024-02-14T17:42:55.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.362+0000 7fdddecd9700 1 -- 172.21.15.178:0/1937121841 >> 172.21.15.178:0/1937121841 conn(0x7fddd8075c00 msgr2=0x7fddd8076010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:55.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.363+0000 7fdddecd9700 1 -- 172.21.15.178:0/1937121841 shutdown_connections 2024-02-14T17:42:55.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.363+0000 7fdddecd9700 1 -- 172.21.15.178:0/1937121841 wait complete. 2024-02-14T17:42:55.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.363+0000 7fdddecd9700 1 Processor -- start 2024-02-14T17:42:55.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.363+0000 7fdddecd9700 1 -- start start 2024-02-14T17:42:55.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.363+0000 7fdddecd9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fddd81acf20 con 0x7fddd8114ab0 2024-02-14T17:42:55.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.363+0000 7fdddecd9700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fddd81ae100 con 0x7fddd8100340 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.363+0000 7fdddecd9700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fddd81af300 con 0x7fddd8100e30 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.363+0000 7fddd7fff700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7fddd8100e30 0x7fddd81a7e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:34478/0 (socket says 172.21.15.178:34478) 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.363+0000 7fddd7fff700 1 -- 172.21.15.178:0/3195847332 learned_addr learned my addr 172.21.15.178:0/3195847332 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413891985 0 0) 0x7fddd81acf20 con 0x7fddd8114ab0 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddc80031f0 con 0x7fddd8114ab0 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854824380 0 0) 0x7fddd81af300 con 0x7fddd8100e30 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddd81acf20 con 0x7fddd8100e30 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412068614 0 0) 0x7fddd81ae100 con 0x7fddd8100340 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddd81af300 con 0x7fddd8100340 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 252958289 0 0) 0x7fddc80031f0 con 0x7fddd8114ab0 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddd81ae100 con 0x7fddd8114ab0 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fddd0001d60 con 0x7fddd8114ab0 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3107019755 0 0) 0x7fddd81acf20 con 0x7fddd8100e30 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddc80031f0 con 0x7fddd8100e30 2024-02-14T17:42:55.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fddc4001870 con 0x7fddd8100e30 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2304932819 0 0) 0x7fddd81af300 con 0x7fddd8100340 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddd81acf20 con 0x7fddd8100340 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fddcc001850 con 0x7fddd8100340 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3875137418 0 0) 0x7fddd81ae100 con 0x7fddd8114ab0 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 >> v1:172.21.15.178:6790/0 conn(0x7fddd8100e30 legacy=0x7fddd81a7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 >> v1:172.21.15.184:6789/0 conn(0x7fddd8100340 legacy=0x7fddd8114200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd81b0500 con 0x7fddd8114ab0 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fdddecd9700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd81af4f0 con 0x7fddd8114ab0 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fddd0004990 con 0x7fddd8114ab0 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fddd0005280 con 0x7fddd8114ab0 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.364+0000 7fdddecd9700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd81af970 con 0x7fddd8114ab0 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.365+0000 7fdddecd9700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.178:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fddd8118a20 con 0x7fddd8114ab0 2024-02-14T17:42:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.366+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7fddd0003a90 con 0x7fddd8114ab0 2024-02-14T17:42:55.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.366+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7fddd0087e30 con 0x7fddd8114ab0 2024-02-14T17:42:55.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.366+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.178:6809/479756253 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fddc807cf40 con 0x7fddc8079230 2024-02-14T17:42:55.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.366+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (unknown 3287965567 0 0) 0x7fddd0088260 con 0x7fddd8114ab0 2024-02-14T17:42:55.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.367+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== osd.2 v1:172.21.15.178:6809/479756253 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fddc807cf40 con 0x7fddc8079230 2024-02-14T17:42:55.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.383+0000 7fdddecd9700 1 -- 172.21.15.178:0/3195847332 --> v1:172.21.15.178:6809/479756253 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fddd81afe70 con 0x7fddc8079230 2024-02-14T17:42:55.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.384+0000 7fddd5ffb700 1 -- 172.21.15.178:0/3195847332 <== osd.2 v1:172.21.15.178:6809/479756253 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3472381573) 0x7fddd81afe70 con 0x7fddc8079230 2024-02-14T17:42:55.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.384+0000 7fdddecd9700 1 -- 172.21.15.178:0/3195847332 >> v1:172.21.15.178:6809/479756253 conn(0x7fddc8079230 legacy=0x7fddc807b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:55.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.384+0000 7fdddecd9700 1 -- 172.21.15.178:0/3195847332 >> v1:172.21.15.178:6800/4205070280 conn(0x7fddc806eaf0 legacy=0x7fddc8070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:55.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.384+0000 7fdddecd9700 1 -- 172.21.15.178:0/3195847332 >> v1:172.21.15.178:6789/0 conn(0x7fddd8114ab0 legacy=0x7fddd81ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:55.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.384+0000 7fdddd276700 1 -- 172.21.15.178:0/3195847332 reap_dead start 2024-02-14T17:42:55.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.384+0000 7fdddecd9700 1 -- 172.21.15.178:0/3195847332 shutdown_connections 2024-02-14T17:42:55.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.384+0000 7fdddecd9700 1 -- 172.21.15.178:0/3195847332 >> 172.21.15.178:0/3195847332 conn(0x7fddd8075c00 msgr2=0x7fddd8106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:55.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.384+0000 7fdddecd9700 1 -- 172.21.15.178:0/3195847332 shutdown_connections 2024-02-14T17:42:55.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:55.384+0000 7fdddecd9700 1 -- 172.21.15.178:0/3195847332 wait complete. 2024-02-14T17:42:56.039 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:42:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:42:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:42:56.069 INFO:teuthology.orchestra.run.smithi178.stdout:111669149730 2024-02-14T17:42:56.069 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-14T17:42:56.685 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:56.686 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:56.730 INFO:teuthology.orchestra.run.smithi178.stdout:77309411369 2024-02-14T17:42:56.731 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-14T17:42:56.849 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:56 smithi178 ceph-mon[30321]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:42:56.849 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:56 smithi178 ceph-mon[39811]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:42:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:56 smithi184 ceph-mon[33536]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:42:57.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.174+0000 7fd1b159e700 1 -- 172.21.15.178:0/154110753 >> v1:172.21.15.178:6790/0 conn(0x7fd1ac1095e0 legacy=0x7fd1ac10b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:57.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.174+0000 7fd1b159e700 1 -- 172.21.15.178:0/154110753 shutdown_connections 2024-02-14T17:42:57.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.174+0000 7fd1b159e700 1 -- 172.21.15.178:0/154110753 >> 172.21.15.178:0/154110753 conn(0x7fd1ac0fc240 msgr2=0x7fd1ac0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:57.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.174+0000 7fd1b159e700 1 -- 172.21.15.178:0/154110753 shutdown_connections 2024-02-14T17:42:57.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.174+0000 7fd1b159e700 1 -- 172.21.15.178:0/154110753 wait complete. 2024-02-14T17:42:57.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.175+0000 7fd1b159e700 1 Processor -- start 2024-02-14T17:42:57.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.175+0000 7fd1b159e700 1 -- start start 2024-02-14T17:42:57.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.175+0000 7fd1b159e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd1ac19d6c0 con 0x7fd1ac1095e0 2024-02-14T17:42:57.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.175+0000 7fd1b159e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd1ac1adfa0 con 0x7fd1ac106730 2024-02-14T17:42:57.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.175+0000 7fd1b159e700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd1ac1af140 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.175+0000 7fd1b0d9d700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7fd1ac10d1b0 0x7fd1ac1ab870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:34490/0 (socket says 172.21.15.178:34490) 2024-02-14T17:42:57.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.175+0000 7fd1b0d9d700 1 -- 172.21.15.178:0/802807308 learned_addr learned my addr 172.21.15.178:0/802807308 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:42:57.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.175+0000 7fd1ab7fe700 1 --1- 172.21.15.178:0/802807308 >> v1:172.21.15.178:6789/0 conn(0x7fd1ac1095e0 0x7fd1ac1a8140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:34326/0 (socket says 172.21.15.178:34326) 2024-02-14T17:42:57.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966061025 0 0) 0x7fd1ac1af140 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1940031f0 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239337184 0 0) 0x7fd1ac19d6c0 con 0x7fd1ac1095e0 2024-02-14T17:42:57.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1ac1af140 con 0x7fd1ac1095e0 2024-02-14T17:42:57.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874870536 0 0) 0x7fd1ac1adfa0 con 0x7fd1ac106730 2024-02-14T17:42:57.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1ac19d6c0 con 0x7fd1ac106730 2024-02-14T17:42:57.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 537732048 0 0) 0x7fd1940031f0 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd1ac1adfa0 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd1a4001c70 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1393245580 0 0) 0x7fd1ac1af140 con 0x7fd1ac1095e0 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd1940031f0 con 0x7fd1ac1095e0 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd1a0001500 con 0x7fd1ac1095e0 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4187431973 0 0) 0x7fd1ac19d6c0 con 0x7fd1ac106730 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd1ac1af140 con 0x7fd1ac106730 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd19c0018d0 con 0x7fd1ac106730 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3613481209 0 0) 0x7fd1ac1adfa0 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 >> v1:172.21.15.184:6789/0 conn(0x7fd1ac106730 legacy=0x7fd1ac10c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 >> v1:172.21.15.178:6789/0 conn(0x7fd1ac1095e0 legacy=0x7fd1ac1a8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:57.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac1b0320 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1b159e700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac1af330 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1b159e700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac1af7b0 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fd1a4003eb0 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.176+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fd1a4005fd0 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.177+0000 7fd1b159e700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.178:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd198000f80 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.178+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7fd1a4003a20 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.178+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7fd1a4088d30 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.178+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.184:6808/2629842997 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd19407cf00 con 0x7fd194079290 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.178+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (unknown 3287965567 0 0) 0x7fd1a4089130 con 0x7fd1ac10d1b0 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.180+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== osd.6 v1:172.21.15.184:6808/2629842997 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fd19407cf00 con 0x7fd194079290 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.197+0000 7fd1b159e700 1 -- 172.21.15.178:0/802807308 --> v1:172.21.15.184:6808/2629842997 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd198002c40 con 0x7fd194079290 2024-02-14T17:42:57.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.197+0000 7fd1a97fa700 1 -- 172.21.15.178:0/802807308 <== osd.6 v1:172.21.15.184:6808/2629842997 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 678733251) 0x7fd198002c40 con 0x7fd194079290 2024-02-14T17:42:57.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.197+0000 7fd1b159e700 1 -- 172.21.15.178:0/802807308 >> v1:172.21.15.184:6808/2629842997 conn(0x7fd194079290 legacy=0x7fd19407b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:57.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.198+0000 7fd1b159e700 1 -- 172.21.15.178:0/802807308 >> v1:172.21.15.178:6800/4205070280 conn(0x7fd19406eac0 legacy=0x7fd194070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:57.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.198+0000 7fd1b159e700 1 -- 172.21.15.178:0/802807308 >> v1:172.21.15.178:6790/0 conn(0x7fd1ac10d1b0 legacy=0x7fd1ac1ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:57.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.198+0000 7fd1b0d9d700 1 -- 172.21.15.178:0/802807308 reap_dead start 2024-02-14T17:42:57.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.198+0000 7fd1b159e700 1 -- 172.21.15.178:0/802807308 shutdown_connections 2024-02-14T17:42:57.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.198+0000 7fd1b159e700 1 -- 172.21.15.178:0/802807308 >> 172.21.15.178:0/802807308 conn(0x7fd1ac0fc240 msgr2=0x7fd1ac0ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:57.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.198+0000 7fd1b159e700 1 -- 172.21.15.178:0/802807308 shutdown_connections 2024-02-14T17:42:57.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:42:57.198+0000 7fd1b159e700 1 -- 172.21.15.178:0/802807308 wait complete. 2024-02-14T17:42:58.346 INFO:teuthology.orchestra.run.smithi178.stdout:55834574895 2024-02-14T17:42:58.347 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-14T17:42:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:58.393 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:42:58 smithi184 ceph-mon[33536]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:42:59.021 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:42:58 smithi178 ceph-mon[39811]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:42:59.022 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:42:58 smithi178 ceph-mon[30321]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:42:59.179 INFO:teuthology.orchestra.run.smithi178.stdout:201863462932 2024-02-14T17:42:59.179 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-14T17:43:00.599 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:00 smithi178 ceph-mon[30321]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:00.861 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:00 smithi184 ceph-mon[33536]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:00.862 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:00 smithi178 ceph-mon[39811]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:01.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:43:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:43:01.889 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:01 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:01.889 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:01 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:01.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:01 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:02 smithi178 ceph-mon[39811]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:02 smithi178 ceph-mon[30321]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:02.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:02 smithi184 ceph-mon[33536]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:03.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.386+0000 7fa025826700 1 -- 172.21.15.178:0/593428549 >> v1:172.21.15.178:6790/0 conn(0x7fa0201097b0 legacy=0x7fa02010bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:03.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.386+0000 7fa025826700 1 -- 172.21.15.178:0/593428549 shutdown_connections 2024-02-14T17:43:03.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.386+0000 7fa025826700 1 -- 172.21.15.178:0/593428549 >> 172.21.15.178:0/593428549 conn(0x7fa0200fc410 msgr2=0x7fa0200fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:03.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.386+0000 7fa025826700 1 -- 172.21.15.178:0/593428549 shutdown_connections 2024-02-14T17:43:03.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.386+0000 7fa025826700 1 -- 172.21.15.178:0/593428549 wait complete. 2024-02-14T17:43:03.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.386+0000 7fa025826700 1 Processor -- start 2024-02-14T17:43:03.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa025826700 1 -- start start 2024-02-14T17:43:03.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa025826700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa0201a69f0 con 0x7fa0201097b0 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa025826700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa0201a7bd0 con 0x7fa020106900 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa025826700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa0201a8dd0 con 0x7fa02010d380 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa01f7fe700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7fa02010d380 0x7fa0201a50b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:34498/0 (socket says 172.21.15.178:34498) 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa01f7fe700 1 -- 172.21.15.178:0/453700463 learned_addr learned my addr 172.21.15.178:0/453700463 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa01e7fc700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa0201097b0 0x7fa0201a1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:34348/0 (socket says 172.21.15.178:34348) 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325913005 0 0) 0x7fa0201a8dd0 con 0x7fa02010d380 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0080031f0 con 0x7fa02010d380 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2027249725 0 0) 0x7fa0201a69f0 con 0x7fa0201097b0 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0201a8dd0 con 0x7fa0201097b0 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398268999 0 0) 0x7fa0201a7bd0 con 0x7fa020106900 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0201a69f0 con 0x7fa020106900 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 983605840 0 0) 0x7fa0080031f0 con 0x7fa02010d380 2024-02-14T17:43:03.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0201a7bd0 con 0x7fa02010d380 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.387+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fa018001bf0 con 0x7fa02010d380 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2582330997 0 0) 0x7fa0201a8dd0 con 0x7fa0201097b0 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0080031f0 con 0x7fa0201097b0 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fa014001500 con 0x7fa0201097b0 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1324141298 0 0) 0x7fa0201a69f0 con 0x7fa020106900 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0201a8dd0 con 0x7fa020106900 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fa0100018d0 con 0x7fa020106900 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 101363539 0 0) 0x7fa0201a7bd0 con 0x7fa02010d380 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 >> v1:172.21.15.184:6789/0 conn(0x7fa020106900 legacy=0x7fa02010cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 >> v1:172.21.15.178:6789/0 conn(0x7fa0201097b0 legacy=0x7fa0201a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0201a9fd0 con 0x7fa02010d380 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa025826700 1 -- 172.21.15.178:0/453700463 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0201a8fc0 con 0x7fa02010d380 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa025826700 1 -- 172.21.15.178:0/453700463 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0201a9440 con 0x7fa02010d380 2024-02-14T17:43:03.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fa018003eb0 con 0x7fa02010d380 2024-02-14T17:43:03.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.388+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fa018005e90 con 0x7fa02010d380 2024-02-14T17:43:03.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.389+0000 7fa025826700 1 -- 172.21.15.178:0/453700463 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00c005120 con 0x7fa02010d380 2024-02-14T17:43:03.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.390+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7fa018003a20 con 0x7fa02010d380 2024-02-14T17:43:03.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.390+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7fa018088ae0 con 0x7fa02010d380 2024-02-14T17:43:03.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.396+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fa018055e60 con 0x7fa02010d380 2024-02-14T17:43:03.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.597+0000 7fa025826700 1 -- 172.21.15.178:0/453700463 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa00c0053d0 con 0x7fa02010d380 2024-02-14T17:43:03.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.597+0000 7fa024824700 1 -- 172.21.15.178:0/453700463 <== mon.2 v1:172.21.15.178:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (unknown 1823589948 0 1336805943) 0x7fa0180541d0 con 0x7fa02010d380 2024-02-14T17:43:03.599 INFO:teuthology.orchestra.run.smithi178.stdout:133143986209 2024-02-14T17:43:03.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.599+0000 7fa025826700 1 -- 172.21.15.178:0/453700463 >> v1:172.21.15.178:6800/4205070280 conn(0x7fa00806ea10 legacy=0x7fa008070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:03.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.599+0000 7fa025826700 1 -- 172.21.15.178:0/453700463 >> v1:172.21.15.178:6790/0 conn(0x7fa02010d380 legacy=0x7fa0201a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:03.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.599+0000 7fa025826700 1 -- 172.21.15.178:0/453700463 shutdown_connections 2024-02-14T17:43:03.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.599+0000 7fa025826700 1 -- 172.21.15.178:0/453700463 >> 172.21.15.178:0/453700463 conn(0x7fa0200fc410 msgr2=0x7fa020100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:03.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.599+0000 7fa025826700 1 -- 172.21.15.178:0/453700463 shutdown_connections 2024-02-14T17:43:03.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:03.599+0000 7fa025826700 1 -- 172.21.15.178:0/453700463 wait complete. 2024-02-14T17:43:03.776 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/453700463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-14T17:43:03.777 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/453700463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-14T17:43:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/453700463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-14T17:43:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.408+0000 7fa80a9a2700 1 -- 172.21.15.178:0/2170048328 >> v1:172.21.15.178:6790/0 conn(0x7fa8041004c0 legacy=0x7fa8041008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.408+0000 7fa80a9a2700 1 -- 172.21.15.178:0/2170048328 shutdown_connections 2024-02-14T17:43:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.408+0000 7fa80a9a2700 1 -- 172.21.15.178:0/2170048328 >> 172.21.15.178:0/2170048328 conn(0x7fa804075bf0 msgr2=0x7fa804076000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.408+0000 7fa80a9a2700 1 -- 172.21.15.178:0/2170048328 shutdown_connections 2024-02-14T17:43:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.408+0000 7fa80a9a2700 1 -- 172.21.15.178:0/2170048328 wait complete. 2024-02-14T17:43:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.409+0000 7fa80a9a2700 1 Processor -- start 2024-02-14T17:43:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.409+0000 7fa80a9a2700 1 -- start start 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.409+0000 7fa80a9a2700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa8041a67d0 con 0x7fa80410a940 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.409+0000 7fa80a9a2700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa8041a79b0 con 0x7fa8041004c0 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.409+0000 7fa80a9a2700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa8041a8bb0 con 0x7fa80410e4b0 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa808f3f700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7fa80410e4b0 0x7fa8041a4f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:39976/0 (socket says 172.21.15.178:39976) 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8037fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa80410a940 0x7fa8041a1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:47696/0 (socket says 172.21.15.178:47696) 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa808f3f700 1 -- 172.21.15.178:0/1208718558 learned_addr learned my addr 172.21.15.178:0/1208718558 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934761648 0 0) 0x7fa8041a8bb0 con 0x7fa80410e4b0 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7ec0035b0 con 0x7fa80410e4b0 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502898870 0 0) 0x7fa8041a67d0 con 0x7fa80410a940 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8041a8bb0 con 0x7fa80410a940 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313305282 0 0) 0x7fa8041a79b0 con 0x7fa8041004c0 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8041a67d0 con 0x7fa8041004c0 2024-02-14T17:43:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 512853317 0 0) 0x7fa7ec0035b0 con 0x7fa80410e4b0 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa8041a79b0 con 0x7fa80410e4b0 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fa7fc001d70 con 0x7fa80410e4b0 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 239828901 0 0) 0x7fa8041a8bb0 con 0x7fa80410a940 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa7ec0035b0 con 0x7fa80410a940 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fa7f80017b0 con 0x7fa80410a940 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1134059370 0 0) 0x7fa8041a79b0 con 0x7fa80410e4b0 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 >> v1:172.21.15.184:6789/0 conn(0x7fa8041004c0 legacy=0x7fa804106e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 >> v1:172.21.15.178:6789/0 conn(0x7fa80410a940 legacy=0x7fa8041a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8041a9db0 con 0x7fa80410e4b0 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa80a9a2700 1 -- 172.21.15.178:0/1208718558 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8041a69c0 con 0x7fa80410e4b0 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa80a9a2700 1 -- 172.21.15.178:0/1208718558 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8041a6ec0 con 0x7fa80410e4b0 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fa7fc002020 con 0x7fa80410e4b0 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.410+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fa7fc006050 con 0x7fa80410e4b0 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.411+0000 7fa80a9a2700 1 -- 172.21.15.178:0/1208718558 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f0005150 con 0x7fa80410e4b0 2024-02-14T17:43:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.412+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7fa7fc01c6e0 con 0x7fa80410e4b0 2024-02-14T17:43:04.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.412+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7fa7fc088ce0 con 0x7fa80410e4b0 2024-02-14T17:43:04.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.417+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fa7fc055fe0 con 0x7fa80410e4b0 2024-02-14T17:43:04.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.611+0000 7f062b165700 1 -- 172.21.15.178:0/31320602 >> v1:172.21.15.178:6789/0 conn(0x7f0624102bc0 legacy=0x7f0624102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:04.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.611+0000 7f062b165700 1 -- 172.21.15.178:0/31320602 shutdown_connections 2024-02-14T17:43:04.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.611+0000 7f062b165700 1 -- 172.21.15.178:0/31320602 >> 172.21.15.178:0/31320602 conn(0x7f06240fc410 msgr2=0x7f06240fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:04.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.611+0000 7f062b165700 1 -- 172.21.15.178:0/31320602 shutdown_connections 2024-02-14T17:43:04.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.611+0000 7f062b165700 1 -- 172.21.15.178:0/31320602 wait complete. 2024-02-14T17:43:04.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.612+0000 7f062b165700 1 Processor -- start 2024-02-14T17:43:04.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.612+0000 7f062b165700 1 -- start start 2024-02-14T17:43:04.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.612+0000 7f062b165700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f06241a8c60 con 0x7f062410a9d0 2024-02-14T17:43:04.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.612+0000 7f062b165700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f06241a9e40 con 0x7f062410e4e0 2024-02-14T17:43:04.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.612+0000 7f062b165700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f06241ab040 con 0x7f0624102bc0 2024-02-14T17:43:04.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.612+0000 7f0628f01700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f0624102bc0 0x7f062410a210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:40002/0 (socket says 172.21.15.178:40002) 2024-02-14T17:43:04.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.612+0000 7f0628f01700 1 -- 172.21.15.178:0/2167882988 learned_addr learned my addr 172.21.15.178:0/2167882988 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:43:04.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470342776 0 0) 0x7f06241a8c60 con 0x7f062410a9d0 2024-02-14T17:43:04.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06140035b0 con 0x7f062410a9d0 2024-02-14T17:43:04.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2274410990 0 0) 0x7f06241a9e40 con 0x7f062410e4e0 2024-02-14T17:43:04.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06241a8c60 con 0x7f062410e4e0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085689998 0 0) 0x7f06241ab040 con 0x7f0624102bc0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06241a9e40 con 0x7f0624102bc0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3015094337 0 0) 0x7f06140035b0 con 0x7f062410a9d0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f06241ab040 con 0x7f062410a9d0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f06100017e0 con 0x7f062410a9d0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3643830322 0 0) 0x7f06241a8c60 con 0x7f062410e4e0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f06140035b0 con 0x7f062410e4e0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2879405885 0 0) 0x7f06241a9e40 con 0x7f0624102bc0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f06241a8c60 con 0x7f0624102bc0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f061c001bb0 con 0x7f062410e4e0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f0618001880 con 0x7f0624102bc0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3844369074 0 0) 0x7f06241ab040 con 0x7f062410a9d0 2024-02-14T17:43:04.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 >> v1:172.21.15.178:6790/0 conn(0x7f0624102bc0 legacy=0x7f062410a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:04.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 >> v1:172.21.15.184:6789/0 conn(0x7f062410e4e0 legacy=0x7f06241a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:04.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06241ac240 con 0x7f062410a9d0 2024-02-14T17:43:04.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f062b165700 1 -- 172.21.15.178:0/2167882988 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06241a8e50 con 0x7f062410a9d0 2024-02-14T17:43:04.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f0610003b10 con 0x7f062410a9d0 2024-02-14T17:43:04.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f0610004a40 con 0x7f062410a9d0 2024-02-14T17:43:04.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.613+0000 7f062b165700 1 -- 172.21.15.178:0/2167882988 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f06241a92d0 con 0x7f062410a9d0 2024-02-14T17:43:04.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.614+0000 7f062b165700 1 -- 172.21.15.178:0/2167882988 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0624069ed0 con 0x7f062410a9d0 2024-02-14T17:43:04.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.615+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f0610003680 con 0x7f062410a9d0 2024-02-14T17:43:04.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.615+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7f06100876e0 con 0x7f062410a9d0 2024-02-14T17:43:04.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.624+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f06100549e0 con 0x7f062410a9d0 2024-02-14T17:43:04.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.625+0000 7fa80a9a2700 1 -- 172.21.15.178:0/1208718558 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa7f0005400 con 0x7fa80410e4b0 2024-02-14T17:43:04.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.625+0000 7fa8017fa700 1 -- 172.21.15.178:0/1208718558 <== mon.2 v1:172.21.15.178:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (unknown 1482059429 0 4067349526) 0x7fa7fc054350 con 0x7fa80410e4b0 2024-02-14T17:43:04.627 INFO:teuthology.orchestra.run.smithi178.stdout:240518168594 2024-02-14T17:43:04.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.628+0000 7fa80a9a2700 1 -- 172.21.15.178:0/1208718558 >> v1:172.21.15.178:6800/4205070280 conn(0x7fa7ec06e820 legacy=0x7fa7ec070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:04.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.628+0000 7fa80a9a2700 1 -- 172.21.15.178:0/1208718558 >> v1:172.21.15.178:6790/0 conn(0x7fa80410e4b0 legacy=0x7fa8041a4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:04.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.629+0000 7fa80a9a2700 1 -- 172.21.15.178:0/1208718558 shutdown_connections 2024-02-14T17:43:04.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.629+0000 7fa80a9a2700 1 -- 172.21.15.178:0/1208718558 >> 172.21.15.178:0/1208718558 conn(0x7fa804075bf0 msgr2=0x7fa804111c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:04.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.629+0000 7fa80a9a2700 1 -- 172.21.15.178:0/1208718558 shutdown_connections 2024-02-14T17:43:04.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.629+0000 7fa80a9a2700 1 -- 172.21.15.178:0/1208718558 wait complete. 2024-02-14T17:43:04.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:04 smithi178 ceph-mon[39811]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:04.741 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:04 smithi178 ceph-mon[30321]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:04.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.837+0000 7f062b165700 1 -- 172.21.15.178:0/2167882988 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0624101d50 con 0x7f062410a9d0 2024-02-14T17:43:04.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.837+0000 7f0621ffb700 1 -- 172.21.15.178:0/2167882988 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 190233549) 0x7f0610052d50 con 0x7f062410a9d0 2024-02-14T17:43:04.839 INFO:teuthology.orchestra.run.smithi178.stdout:34359738421 2024-02-14T17:43:04.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.840+0000 7f062b165700 1 -- 172.21.15.178:0/2167882988 >> v1:172.21.15.178:6800/4205070280 conn(0x7f061406eb70 legacy=0x7f0614071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:04.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.840+0000 7f062b165700 1 -- 172.21.15.178:0/2167882988 >> v1:172.21.15.178:6789/0 conn(0x7f062410a9d0 legacy=0x7f06241a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:04.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.840+0000 7f062b165700 1 -- 172.21.15.178:0/2167882988 shutdown_connections 2024-02-14T17:43:04.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.840+0000 7f062b165700 1 -- 172.21.15.178:0/2167882988 >> 172.21.15.178:0/2167882988 conn(0x7f06240fc410 msgr2=0x7f0624110910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:04.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.840+0000 7f062b165700 1 -- 172.21.15.178:0/2167882988 shutdown_connections 2024-02-14T17:43:04.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:04.840+0000 7f062b165700 1 -- 172.21.15.178:0/2167882988 wait complete. 2024-02-14T17:43:04.871 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:04 smithi184 ceph-mon[33536]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:05.245 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:43:05.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.351+0000 7fafd354b700 1 -- 172.21.15.178:0/113653063 >> v1:172.21.15.178:6789/0 conn(0x7fafcc10e490 legacy=0x7fafcc110950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:05.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.351+0000 7fafd354b700 1 -- 172.21.15.178:0/113653063 shutdown_connections 2024-02-14T17:43:05.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.351+0000 7fafd354b700 1 -- 172.21.15.178:0/113653063 >> 172.21.15.178:0/113653063 conn(0x7fafcc0fd520 msgr2=0x7fafcc0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:05.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.351+0000 7fafd354b700 1 -- 172.21.15.178:0/113653063 shutdown_connections 2024-02-14T17:43:05.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.351+0000 7fafd354b700 1 -- 172.21.15.178:0/113653063 wait complete. 2024-02-14T17:43:05.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.352+0000 7fafd354b700 1 Processor -- start 2024-02-14T17:43:05.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.352+0000 7fafd354b700 1 -- start start 2024-02-14T17:43:05.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.352+0000 7fafd354b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fafcc1a68d0 con 0x7fafcc10e490 2024-02-14T17:43:05.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.352+0000 7fafd354b700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fafcc1a7ad0 con 0x7fafcc107a10 2024-02-14T17:43:05.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.352+0000 7fafd354b700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fafcc1a8cd0 con 0x7fafcc10a8c0 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.352+0000 7fafd0ae6700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7fafcc10a8c0 0x7fafcc1a1840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:40006/0 (socket says 172.21.15.178:40006) 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.352+0000 7fafd0ae6700 1 -- 172.21.15.178:0/995784352 learned_addr learned my addr 172.21.15.178:0/995784352 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.352+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028923939 0 0) 0x7fafcc1a8cd0 con 0x7fafcc10a8c0 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.352+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafb80049e0 con 0x7fafcc10a8c0 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.352+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641525736 0 0) 0x7fafcc1a7ad0 con 0x7fafcc107a10 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafcc1a8cd0 con 0x7fafcc107a10 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216205699 0 0) 0x7fafcc1a68d0 con 0x7fafcc10e490 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafcc1a7ad0 con 0x7fafcc10e490 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2310077030 0 0) 0x7fafb80049e0 con 0x7fafcc10a8c0 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fafcc1a68d0 con 0x7fafcc10a8c0 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fafbc001930 con 0x7fafcc10a8c0 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3991862437 0 0) 0x7fafcc1a7ad0 con 0x7fafcc10e490 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fafb80049e0 con 0x7fafcc10e490 2024-02-14T17:43:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1881574196 0 0) 0x7fafcc1a8cd0 con 0x7fafcc107a10 2024-02-14T17:43:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fafcc1a7ad0 con 0x7fafcc107a10 2024-02-14T17:43:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fafc4001c20 con 0x7fafcc10e490 2024-02-14T17:43:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fafc8001780 con 0x7fafcc107a10 2024-02-14T17:43:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 791280414 0 0) 0x7fafcc1a68d0 con 0x7fafcc10a8c0 2024-02-14T17:43:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 >> v1:172.21.15.184:6789/0 conn(0x7fafcc107a10 legacy=0x7fafcc10daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 >> v1:172.21.15.178:6789/0 conn(0x7fafcc10e490 legacy=0x7fafcc1a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc1a9ed0 con 0x7fafcc10a8c0 2024-02-14T17:43:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafd354b700 1 -- 172.21.15.178:0/995784352 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc1a7cc0 con 0x7fafcc10a8c0 2024-02-14T17:43:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.353+0000 7fafd354b700 1 -- 172.21.15.178:0/995784352 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc1a8140 con 0x7fafcc10a8c0 2024-02-14T17:43:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.354+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fafbc001cc0 con 0x7fafcc10a8c0 2024-02-14T17:43:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.354+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fafbc005d70 con 0x7fafcc10a8c0 2024-02-14T17:43:05.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.354+0000 7fafd354b700 1 -- 172.21.15.178:0/995784352 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb0005150 con 0x7fafcc10a8c0 2024-02-14T17:43:05.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.355+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7fafbc006f40 con 0x7fafcc10a8c0 2024-02-14T17:43:05.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.356+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7fafbc088960 con 0x7fafcc10a8c0 2024-02-14T17:43:05.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.362+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fafbc055e30 con 0x7fafcc10a8c0 2024-02-14T17:43:05.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.565+0000 7fafd354b700 1 -- 172.21.15.178:0/995784352 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fafb0005400 con 0x7fafcc10a8c0 2024-02-14T17:43:05.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.565+0000 7fafc27fc700 1 -- 172.21.15.178:0/995784352 <== mon.2 v1:172.21.15.178:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (unknown 2131975755 0 270668312) 0x7fafbc0541a0 con 0x7fafcc10a8c0 2024-02-14T17:43:05.567 INFO:teuthology.orchestra.run.smithi178.stdout:167503724571 2024-02-14T17:43:05.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.567+0000 7fafd354b700 1 -- 172.21.15.178:0/995784352 >> v1:172.21.15.178:6800/4205070280 conn(0x7fafb806eab0 legacy=0x7fafb8070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:05.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.567+0000 7fafd354b700 1 -- 172.21.15.178:0/995784352 >> v1:172.21.15.178:6790/0 conn(0x7fafcc10a8c0 legacy=0x7fafcc1a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:05.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.567+0000 7fafd354b700 1 -- 172.21.15.178:0/995784352 shutdown_connections 2024-02-14T17:43:05.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.567+0000 7fafd354b700 1 -- 172.21.15.178:0/995784352 >> 172.21.15.178:0/995784352 conn(0x7fafcc0fd520 msgr2=0x7fafcc111120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:05.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.567+0000 7fafd354b700 1 -- 172.21.15.178:0/995784352 shutdown_connections 2024-02-14T17:43:05.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:05.567+0000 7fafd354b700 1 -- 172.21.15.178:0/995784352 wait complete. 2024-02-14T17:43:05.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:43:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:43:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1208718558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-14T17:43:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2167882988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T17:43:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/995784352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-14T17:43:05.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1208718558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-14T17:43:05.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2167882988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T17:43:05.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/995784352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-14T17:43:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1208718558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-14T17:43:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2167882988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T17:43:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/995784352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-14T17:43:06.278 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986206 got 133143986209 for osd.4 2024-02-14T17:43:06.279 DEBUG:teuthology.parallel:result is None 2024-02-14T17:43:06.829 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:06 smithi178 ceph-mon[30321]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:43:06.829 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:06 smithi178 ceph-mon[39811]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:43:06.857 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168592 got 240518168594 for osd.7 2024-02-14T17:43:06.857 DEBUG:teuthology.parallel:result is None 2024-02-14T17:43:06.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:06 smithi184 ceph-mon[33536]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:43:07.340 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:43:07.716 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:43:07.724 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:43:07.750 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724568 got 167503724571 for osd.5 2024-02-14T17:43:07.750 DEBUG:teuthology.parallel:result is None 2024-02-14T17:43:08.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:07 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:07 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:07 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:07 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:07 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:07 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.696 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738421 for osd.0 2024-02-14T17:43:08.696 DEBUG:teuthology.parallel:result is None 2024-02-14T17:43:08.937 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:08.937 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.937 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.937 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.937 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.937 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.937 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.937 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.939 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.939 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.939 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.939 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.939 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.939 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:08.939 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:08.939 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:08 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:08 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:10.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:09 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi178 to 3760M 2024-02-14T17:43:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:09 smithi184 ceph-mon[33536]: Updating smithi178:/etc/ceph/ceph.conf 2024-02-14T17:43:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:09 smithi184 ceph-mon[33536]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-14T17:43:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:09 smithi184 ceph-mon[33536]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:43:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:09 smithi184 ceph-mon[33536]: Updating smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:43:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:09 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:09 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:09 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:09 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:09 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:09 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.222 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi178 to 3760M 2024-02-14T17:43:10.222 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[39811]: Updating smithi178:/etc/ceph/ceph.conf 2024-02-14T17:43:10.222 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[39811]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-14T17:43:10.222 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[39811]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[39811]: Updating smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi178 to 3760M 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[30321]: Updating smithi178:/etc/ceph/ceph.conf 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[30321]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[30321]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[30321]: Updating smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.223 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.224 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.224 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:10.224 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:09 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:11.179 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[39811]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:43:11.179 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[39811]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:43:11.179 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[39811]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:11.179 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[39811]: Updating smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:43:11.179 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[39811]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:43:11.179 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[39811]: Deploying daemon alertmanager.a on smithi178 2024-02-14T17:43:11.180 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[30321]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:43:11.180 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[30321]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:43:11.180 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[30321]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:11.180 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[30321]: Updating smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:43:11.180 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[30321]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:43:11.180 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:10 smithi178 ceph-mon[30321]: Deploying daemon alertmanager.a on smithi178 2024-02-14T17:43:11.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:43:10 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:43:11.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:10 smithi184 ceph-mon[33536]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:43:11.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:10 smithi184 ceph-mon[33536]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:43:11.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:10 smithi184 ceph-mon[33536]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:11.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:10 smithi184 ceph-mon[33536]: Updating smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:43:11.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:10 smithi184 ceph-mon[33536]: Updating smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:43:11.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:10 smithi184 ceph-mon[33536]: Deploying daemon alertmanager.a on smithi178 2024-02-14T17:43:11.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.279+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3955860483 >> v1:172.21.15.178:6790/0 conn(0x7fa6f410a9f0 legacy=0x7fa6f410cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:11.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.279+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3955860483 shutdown_connections 2024-02-14T17:43:11.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.279+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3955860483 >> 172.21.15.178:0/3955860483 conn(0x7fa6f40fc430 msgr2=0x7fa6f40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:11.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.280+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3955860483 shutdown_connections 2024-02-14T17:43:11.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.280+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3955860483 wait complete. 2024-02-14T17:43:11.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.280+0000 7fa6fc06d700 1 Processor -- start 2024-02-14T17:43:11.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.280+0000 7fa6fc06d700 1 -- start start 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.280+0000 7fa6fc06d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa6f41a6810 con 0x7fa6f410e500 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.280+0000 7fa6fc06d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa6f41a79f0 con 0x7fa6f410a9f0 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.280+0000 7fa6fc06d700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa6f41a8bf0 con 0x7fa6f4102be0 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6f9e09700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7fa6f4102be0 0x7fa6f410a030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:40018/0 (socket says 172.21.15.178:40018) 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6f9e09700 1 -- 172.21.15.178:0/3230150885 learned_addr learned my addr 172.21.15.178:0/3230150885 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398744307 0 0) 0x7fa6f41a6810 con 0x7fa6f410e500 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e00031f0 con 0x7fa6f410e500 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933007427 0 0) 0x7fa6f41a8bf0 con 0x7fa6f4102be0 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f41a6810 con 0x7fa6f4102be0 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 40284524 0 0) 0x7fa6f41a6810 con 0x7fa6f4102be0 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6f41a8bf0 con 0x7fa6f4102be0 2024-02-14T17:43:11.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2009991340 0 0) 0x7fa6e00031f0 con 0x7fa6f410e500 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6f41a6810 con 0x7fa6f410e500 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fa6f00018d0 con 0x7fa6f4102be0 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831331709 0 0) 0x7fa6f41a79f0 con 0x7fa6f410a9f0 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e00031f0 con 0x7fa6f410a9f0 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fa6ec001af0 con 0x7fa6f410e500 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2096370497 0 0) 0x7fa6f41a8bf0 con 0x7fa6f4102be0 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 >> v1:172.21.15.184:6789/0 conn(0x7fa6f410a9f0 legacy=0x7fa6f41a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 >> v1:172.21.15.178:6789/0 conn(0x7fa6f410e500 legacy=0x7fa6f41a4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f41a9df0 con 0x7fa6f4102be0 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3230150885 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f41a6a00 con 0x7fa6f4102be0 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.281+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3230150885 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f41a6e80 con 0x7fa6f4102be0 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.282+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fa6f0001d70 con 0x7fa6f4102be0 2024-02-14T17:43:11.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.282+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fa6f0005ab0 con 0x7fa6f4102be0 2024-02-14T17:43:11.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.282+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3230150885 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d8005120 con 0x7fa6f4102be0 2024-02-14T17:43:11.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.283+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7fa6f0003680 con 0x7fa6f4102be0 2024-02-14T17:43:11.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.283+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7fa6f0088800 con 0x7fa6f4102be0 2024-02-14T17:43:11.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.288+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fa6f0055ae0 con 0x7fa6f4102be0 2024-02-14T17:43:11.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.499+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3230150885 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa6d80053d0 con 0x7fa6f4102be0 2024-02-14T17:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.499+0000 7fa6eaffd700 1 -- 172.21.15.178:0/3230150885 <== mon.2 v1:172.21.15.178:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (unknown 383520633 0 1195175880) 0x7fa6f0053e50 con 0x7fa6f4102be0 2024-02-14T17:43:11.502 INFO:teuthology.orchestra.run.smithi178.stdout:111669149733 2024-02-14T17:43:11.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.501+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3230150885 >> v1:172.21.15.178:6800/4205070280 conn(0x7fa6e006ea90 legacy=0x7fa6e0070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:11.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.501+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3230150885 >> v1:172.21.15.178:6790/0 conn(0x7fa6f4102be0 legacy=0x7fa6f410a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:11.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.501+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3230150885 shutdown_connections 2024-02-14T17:43:11.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.501+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3230150885 >> 172.21.15.178:0/3230150885 conn(0x7fa6f40fc430 msgr2=0x7fa6f40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:11.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.501+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3230150885 shutdown_connections 2024-02-14T17:43:11.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:11.501+0000 7fa6fc06d700 1 -- 172.21.15.178:0/3230150885 wait complete. 2024-02-14T17:43:12.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:11 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:12.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3230150885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-14T17:43:12.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:11 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:12.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3230150885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-14T17:43:12.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:11 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:12.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3230150885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-14T17:43:12.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.716+0000 7f02cb6ba700 1 -- 172.21.15.178:0/730383530 >> v1:172.21.15.178:6790/0 conn(0x7f02c410d380 legacy=0x7f02c410f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:12.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.716+0000 7f02cb6ba700 1 -- 172.21.15.178:0/730383530 shutdown_connections 2024-02-14T17:43:12.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.716+0000 7f02cb6ba700 1 -- 172.21.15.178:0/730383530 >> 172.21.15.178:0/730383530 conn(0x7f02c40fc410 msgr2=0x7f02c40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:12.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.716+0000 7f02cb6ba700 1 -- 172.21.15.178:0/730383530 shutdown_connections 2024-02-14T17:43:12.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.716+0000 7f02cb6ba700 1 -- 172.21.15.178:0/730383530 wait complete. 2024-02-14T17:43:12.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.717+0000 7f02cb6ba700 1 Processor -- start 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.717+0000 7f02cb6ba700 1 -- start start 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.717+0000 7f02cb6ba700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f02c41a6970 con 0x7f02c4106900 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.717+0000 7f02cb6ba700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f02c41a7b50 con 0x7f02c41097b0 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.717+0000 7f02cb6ba700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f02c41a8d50 con 0x7f02c410d380 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.717+0000 7f02c9456700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f02c4106900 0x7f02c410cb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:47766/0 (socket says 172.21.15.178:47766) 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.717+0000 7f02c9456700 1 -- 172.21.15.178:0/4147105562 learned_addr learned my addr 172.21.15.178:0/4147105562 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884554464 0 0) 0x7f02c41a6970 con 0x7f02c4106900 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02b00035b0 con 0x7f02c4106900 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991144402 0 0) 0x7f02c41a8d50 con 0x7f02c410d380 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02c41a6970 con 0x7f02c410d380 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622741164 0 0) 0x7f02c41a7b50 con 0x7f02c41097b0 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02c41a8d50 con 0x7f02c41097b0 2024-02-14T17:43:12.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4279407548 0 0) 0x7f02b00035b0 con 0x7f02c4106900 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f02c41a7b50 con 0x7f02c4106900 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f02b4001930 con 0x7f02c4106900 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1121889377 0 0) 0x7f02c41a6970 con 0x7f02c410d380 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f02b00035b0 con 0x7f02c410d380 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f02bc001bc0 con 0x7f02c410d380 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1458920460 0 0) 0x7f02c41a8d50 con 0x7f02c41097b0 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f02c41a6970 con 0x7f02c41097b0 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f02c0001780 con 0x7f02c41097b0 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2746444125 0 0) 0x7f02b00035b0 con 0x7f02c410d380 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 >> v1:172.21.15.184:6789/0 conn(0x7f02c41097b0 legacy=0x7f02c41a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 >> v1:172.21.15.178:6789/0 conn(0x7f02c4106900 legacy=0x7f02c410cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c41a9f50 con 0x7f02c410d380 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02cb6ba700 1 -- 172.21.15.178:0/4147105562 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c41a8f40 con 0x7f02c410d380 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02cb6ba700 1 -- 172.21.15.178:0/4147105562 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c41a93c0 con 0x7f02c410d380 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f02bc004a60 con 0x7f02c410d380 2024-02-14T17:43:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.718+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f02bc004fc0 con 0x7f02c410d380 2024-02-14T17:43:12.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.719+0000 7f02cb6ba700 1 -- 172.21.15.178:0/4147105562 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a8005120 con 0x7f02c410d380 2024-02-14T17:43:12.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.720+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f02bc003b70 con 0x7f02c410d380 2024-02-14T17:43:12.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.720+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7f02bc087c20 con 0x7f02c410d380 2024-02-14T17:43:12.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.725+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f02bc054f00 con 0x7f02c410d380 2024-02-14T17:43:12.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.923+0000 7f02cb6ba700 1 -- 172.21.15.178:0/4147105562 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f02a80053d0 con 0x7f02c410d380 2024-02-14T17:43:12.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.923+0000 7f02ba7fc700 1 -- 172.21.15.178:0/4147105562 <== mon.2 v1:172.21.15.178:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (unknown 1274345170 0 3246014696) 0x7f02bc053270 con 0x7f02c410d380 2024-02-14T17:43:12.926 INFO:teuthology.orchestra.run.smithi178.stdout:201863462935 2024-02-14T17:43:12.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.926+0000 7f02cb6ba700 1 -- 172.21.15.178:0/4147105562 >> v1:172.21.15.178:6800/4205070280 conn(0x7f02b0077240 legacy=0x7f02b0079700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:12.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.926+0000 7f02cb6ba700 1 -- 172.21.15.178:0/4147105562 >> v1:172.21.15.178:6790/0 conn(0x7f02c410d380 legacy=0x7f02c41a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:12.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.926+0000 7f02cb6ba700 1 -- 172.21.15.178:0/4147105562 shutdown_connections 2024-02-14T17:43:12.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.926+0000 7f02cb6ba700 1 -- 172.21.15.178:0/4147105562 >> 172.21.15.178:0/4147105562 conn(0x7f02c40fc410 msgr2=0x7f02c40ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:12.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.926+0000 7f02cb6ba700 1 -- 172.21.15.178:0/4147105562 shutdown_connections 2024-02-14T17:43:12.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:12.926+0000 7f02cb6ba700 1 -- 172.21.15.178:0/4147105562 wait complete. 2024-02-14T17:43:13.016 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:12 smithi178 ceph-mon[39811]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:13.016 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:12 smithi178 ceph-mon[30321]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:13.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:12 smithi184 ceph-mon[33536]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:13.553 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149730 got 111669149733 for osd.3 2024-02-14T17:43:13.553 DEBUG:teuthology.parallel:result is None 2024-02-14T17:43:14.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4147105562' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-14T17:43:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4147105562' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-14T17:43:14.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4147105562' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-14T17:43:14.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.494+0000 7fce8b7c0700 1 -- 172.21.15.178:0/3151249768 >> v1:172.21.15.178:6789/0 conn(0x7fce8410d3b0 legacy=0x7fce8410f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:14.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.494+0000 7fce8b7c0700 1 -- 172.21.15.178:0/3151249768 shutdown_connections 2024-02-14T17:43:14.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.494+0000 7fce8b7c0700 1 -- 172.21.15.178:0/3151249768 >> 172.21.15.178:0/3151249768 conn(0x7fce84075c60 msgr2=0x7fce84076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:14.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.494+0000 7fce8b7c0700 1 -- 172.21.15.178:0/3151249768 shutdown_connections 2024-02-14T17:43:14.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.494+0000 7fce8b7c0700 1 -- 172.21.15.178:0/3151249768 wait complete. 2024-02-14T17:43:14.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce8b7c0700 1 Processor -- start 2024-02-14T17:43:14.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce8b7c0700 1 -- start start 2024-02-14T17:43:14.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce8b7c0700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fce841a67a0 con 0x7fce84109810 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce8b7c0700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fce841a7980 con 0x7fce8410d3b0 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce8b7c0700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fce841a8b80 con 0x7fce84101550 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce8955c700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7fce84101550 0x7fce841008d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:41422/0 (socket says 172.21.15.178:41422) 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce88d5b700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fce84109810 0x7fce841a1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:41838/0 (socket says 172.21.15.178:41838) 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce8955c700 1 -- 172.21.15.178:0/2671990254 learned_addr learned my addr 172.21.15.178:0/2671990254 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331364550 0 0) 0x7fce841a67a0 con 0x7fce84109810 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce700035b0 con 0x7fce84109810 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196530499 0 0) 0x7fce841a8b80 con 0x7fce84101550 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce841a67a0 con 0x7fce84101550 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131391032 0 0) 0x7fce841a7980 con 0x7fce8410d3b0 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce841a8b80 con 0x7fce8410d3b0 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1055532556 0 0) 0x7fce700035b0 con 0x7fce84109810 2024-02-14T17:43:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce841a7980 con 0x7fce84109810 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fce80001890 con 0x7fce84109810 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.495+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2776419154 0 0) 0x7fce841a67a0 con 0x7fce84101550 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.496+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce700035b0 con 0x7fce84101550 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.496+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fce740019d0 con 0x7fce84101550 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.496+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3805905228 0 0) 0x7fce841a7980 con 0x7fce84109810 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.496+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 >> v1:172.21.15.178:6790/0 conn(0x7fce84101550 legacy=0x7fce841008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.496+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 >> v1:172.21.15.184:6789/0 conn(0x7fce8410d3b0 legacy=0x7fce841a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.496+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce841a9d80 con 0x7fce84109810 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.496+0000 7fce8b7c0700 1 -- 172.21.15.178:0/2671990254 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce841a8d70 con 0x7fce84109810 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.496+0000 7fce8b7c0700 1 -- 172.21.15.178:0/2671990254 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fce841a91f0 con 0x7fce84109810 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.496+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fce80003b30 con 0x7fce84109810 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.496+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fce80005b70 con 0x7fce84109810 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.497+0000 7fce8b7c0700 1 -- 172.21.15.178:0/2671990254 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce68005120 con 0x7fce84109810 2024-02-14T17:43:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.497+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7fce800036a0 con 0x7fce84109810 2024-02-14T17:43:14.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.498+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7fce800888c0 con 0x7fce84109810 2024-02-14T17:43:14.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.502+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fce80055ba0 con 0x7fce84109810 2024-02-14T17:43:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.704+0000 7fce8b7c0700 1 -- 172.21.15.178:0/2671990254 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fce680053d0 con 0x7fce84109810 2024-02-14T17:43:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.704+0000 7fce7a7fc700 1 -- 172.21.15.178:0/2671990254 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 3109548707) 0x7fce80053f10 con 0x7fce84109810 2024-02-14T17:43:14.706 INFO:teuthology.orchestra.run.smithi178.stdout:55834574899 2024-02-14T17:43:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.706+0000 7fce8b7c0700 1 -- 172.21.15.178:0/2671990254 >> v1:172.21.15.178:6800/4205070280 conn(0x7fce7006e740 legacy=0x7fce70070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.706+0000 7fce8b7c0700 1 -- 172.21.15.178:0/2671990254 >> v1:172.21.15.178:6789/0 conn(0x7fce84109810 legacy=0x7fce841a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.706+0000 7fce8b7c0700 1 -- 172.21.15.178:0/2671990254 shutdown_connections 2024-02-14T17:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.706+0000 7fce8b7c0700 1 -- 172.21.15.178:0/2671990254 >> 172.21.15.178:0/2671990254 conn(0x7fce84075c60 msgr2=0x7fce841108b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.707+0000 7fce8b7c0700 1 -- 172.21.15.178:0/2671990254 shutdown_connections 2024-02-14T17:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:14.707+0000 7fce8b7c0700 1 -- 172.21.15.178:0/2671990254 wait complete. 2024-02-14T17:43:15.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:14 smithi184 ceph-mon[33536]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2671990254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T17:43:15.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:14 smithi178 ceph-mon[39811]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:15.289 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2671990254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T17:43:15.289 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:14 smithi178 ceph-mon[30321]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:15.289 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2671990254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T17:43:15.658 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574895 got 55834574899 for osd.1 2024-02-14T17:43:15.658 DEBUG:teuthology.parallel:result is None 2024-02-14T17:43:15.922 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:43:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:43:16.031 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462932 got 201863462935 for osd.6 2024-02-14T17:43:16.032 DEBUG:teuthology.parallel:result is None 2024-02-14T17:43:16.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.263+0000 7f715abc1700 1 -- 172.21.15.178:0/424496012 >> v1:172.21.15.178:6790/0 conn(0x7f7154106900 legacy=0x7f7154106ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.263+0000 7f715abc1700 1 -- 172.21.15.178:0/424496012 shutdown_connections 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.263+0000 7f715abc1700 1 -- 172.21.15.178:0/424496012 >> 172.21.15.178:0/424496012 conn(0x7f71540fc410 msgr2=0x7f71540fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.263+0000 7f715abc1700 1 -- 172.21.15.178:0/424496012 shutdown_connections 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.263+0000 7f715abc1700 1 -- 172.21.15.178:0/424496012 wait complete. 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f715abc1700 1 Processor -- start 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f715abc1700 1 -- start start 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f715abc1700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f71541a67f0 con 0x7f715410d380 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f715abc1700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f71541a79d0 con 0x7f7154106900 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f715abc1700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f71541a8bd0 con 0x7f71541097b0 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f7153fff700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f71541097b0 0x7f71541a1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:41440/0 (socket says 172.21.15.178:41440) 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f7153fff700 1 -- 172.21.15.178:0/2876108070 learned_addr learned my addr 172.21.15.178:0/2876108070 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:43:16.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561018952 0 0) 0x7f71541a8bd0 con 0x7f71541097b0 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71440031f0 con 0x7f71541097b0 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998745773 0 0) 0x7f71541a67f0 con 0x7f715410d380 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71541a8bd0 con 0x7f715410d380 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304577169 0 0) 0x7f71541a79d0 con 0x7f7154106900 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.264+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71541a67f0 con 0x7f7154106900 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1593190486 0 0) 0x7f71440031f0 con 0x7f71541097b0 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f71541a79d0 con 0x7f71541097b0 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f7140001650 con 0x7f71541097b0 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2998991668 0 0) 0x7f71541a8bd0 con 0x7f715410d380 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f71440031f0 con 0x7f715410d380 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f714c001b30 con 0x7f715410d380 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4122428657 0 0) 0x7f71541a67f0 con 0x7f7154106900 2024-02-14T17:43:16.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f71541a8bd0 con 0x7f7154106900 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f7148001960 con 0x7f7154106900 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1102854922 0 0) 0x7f71541a79d0 con 0x7f71541097b0 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 >> v1:172.21.15.184:6789/0 conn(0x7f7154106900 legacy=0x7f715410ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 >> v1:172.21.15.178:6789/0 conn(0x7f715410d380 legacy=0x7f71541a4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71541a9dd0 con 0x7f71541097b0 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f715abc1700 1 -- 172.21.15.178:0/2876108070 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71541a69e0 con 0x7f71541097b0 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f7140003b10 con 0x7f71541097b0 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f7140005b50 con 0x7f71541097b0 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.265+0000 7f715abc1700 1 -- 172.21.15.178:0/2876108070 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f71541a6e60 con 0x7f71541097b0 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.266+0000 7f715abc1700 1 -- 172.21.15.178:0/2876108070 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7154102040 con 0x7f71541097b0 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.266+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f7140005cb0 con 0x7f71541097b0 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.267+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7f71400888d0 con 0x7f71541097b0 2024-02-14T17:43:16.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.271+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f7140055b70 con 0x7f71541097b0 2024-02-14T17:43:16.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.476+0000 7f715abc1700 1 -- 172.21.15.178:0/2876108070 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f715410c050 con 0x7f71541097b0 2024-02-14T17:43:16.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.476+0000 7f7151ffb700 1 -- 172.21.15.178:0/2876108070 <== mon.2 v1:172.21.15.178:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 3275464226) 0x7f7140053ee0 con 0x7f71541097b0 2024-02-14T17:43:16.478 INFO:teuthology.orchestra.run.smithi178.stdout:77309411373 2024-02-14T17:43:16.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.479+0000 7f715abc1700 1 -- 172.21.15.178:0/2876108070 >> v1:172.21.15.178:6800/4205070280 conn(0x7f714406ea80 legacy=0x7f7144070f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:16.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.479+0000 7f715abc1700 1 -- 172.21.15.178:0/2876108070 >> v1:172.21.15.178:6790/0 conn(0x7f71541097b0 legacy=0x7f71541a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:16.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.479+0000 7f715abc1700 1 -- 172.21.15.178:0/2876108070 shutdown_connections 2024-02-14T17:43:16.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.479+0000 7f715abc1700 1 -- 172.21.15.178:0/2876108070 >> 172.21.15.178:0/2876108070 conn(0x7f71540fc410 msgr2=0x7f7154110010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:16.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.479+0000 7f715abc1700 1 -- 172.21.15.178:0/2876108070 shutdown_connections 2024-02-14T17:43:16.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:16.479+0000 7f715abc1700 1 -- 172.21.15.178:0/2876108070 wait complete. 2024-02-14T17:43:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:16 smithi184 ceph-mon[33536]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2876108070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T17:43:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:16 smithi178 ceph-mon[39811]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:17.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2876108070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T17:43:17.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:16 smithi178 ceph-mon[30321]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:17.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2876108070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T17:43:17.597 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411369 got 77309411373 for osd.2 2024-02-14T17:43:17.597 DEBUG:teuthology.parallel:result is None 2024-02-14T17:43:17.597 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-14T17:43:17.597 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-14T17:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:43:19.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:18 smithi178 ceph-mon[30321]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:18 smithi178 ceph-mon[39811]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:19.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:18 smithi184 ceph-mon[33536]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:20.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.802+0000 7ffb8548b700 1 -- 172.21.15.178:0/1926835548 >> v1:172.21.15.178:6789/0 conn(0x7ffb8010d380 legacy=0x7ffb8010f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:20.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.802+0000 7ffb8548b700 1 -- 172.21.15.178:0/1926835548 shutdown_connections 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.803+0000 7ffb8548b700 1 -- 172.21.15.178:0/1926835548 >> 172.21.15.178:0/1926835548 conn(0x7ffb800fc410 msgr2=0x7ffb800fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.803+0000 7ffb8548b700 1 -- 172.21.15.178:0/1926835548 shutdown_connections 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.803+0000 7ffb8548b700 1 -- 172.21.15.178:0/1926835548 wait complete. 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.803+0000 7ffb8548b700 1 Processor -- start 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.803+0000 7ffb8548b700 1 -- start start 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb8548b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb801a57a0 con 0x7ffb8010d380 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb8548b700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb801a6980 con 0x7ffb80106900 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb8548b700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb801a7b80 con 0x7ffb801097b0 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb7e7fc700 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7ffb801097b0 0x7ffb801a0730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:41458/0 (socket says 172.21.15.178:41458) 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb7f7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ffb8010d380 0x7ffb801a3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:41872/0 (socket says 172.21.15.178:41872) 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb7e7fc700 1 -- 172.21.15.178:0/3236966946 learned_addr learned my addr 172.21.15.178:0/3236966946 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720601718 0 0) 0x7ffb801a57a0 con 0x7ffb8010d380 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb680035b0 con 0x7ffb8010d380 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413696023 0 0) 0x7ffb801a7b80 con 0x7ffb801097b0 2024-02-14T17:43:20.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb801a57a0 con 0x7ffb801097b0 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243397776 0 0) 0x7ffb801a6980 con 0x7ffb80106900 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb801a7b80 con 0x7ffb80106900 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 358956328 0 0) 0x7ffb680035b0 con 0x7ffb8010d380 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb801a6980 con 0x7ffb8010d380 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ffb78004630 con 0x7ffb8010d380 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 895673587 0 0) 0x7ffb801a57a0 con 0x7ffb801097b0 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb680035b0 con 0x7ffb801097b0 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ffb74001870 con 0x7ffb801097b0 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2525235472 0 0) 0x7ffb801a7b80 con 0x7ffb80106900 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb801a57a0 con 0x7ffb80106900 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.804+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ffb700019f0 con 0x7ffb80106900 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.805+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2345501676 0 0) 0x7ffb680035b0 con 0x7ffb801097b0 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.805+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 >> v1:172.21.15.184:6789/0 conn(0x7ffb80106900 legacy=0x7ffb8010ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:20.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.805+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 >> v1:172.21.15.178:6789/0 conn(0x7ffb8010d380 legacy=0x7ffb801a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:20.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.805+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb801a8d80 con 0x7ffb801097b0 2024-02-14T17:43:20.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.805+0000 7ffb8548b700 1 -- 172.21.15.178:0/3236966946 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb801a7d70 con 0x7ffb801097b0 2024-02-14T17:43:20.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.805+0000 7ffb8548b700 1 -- 172.21.15.178:0/3236966946 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb801a81f0 con 0x7ffb801097b0 2024-02-14T17:43:20.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.805+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7ffb74003b30 con 0x7ffb801097b0 2024-02-14T17:43:20.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.805+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ffb74005bc0 con 0x7ffb801097b0 2024-02-14T17:43:20.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.805+0000 7ffb8548b700 1 -- 172.21.15.178:0/3236966946 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c005150 con 0x7ffb801097b0 2024-02-14T17:43:20.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.806+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7ffb740036a0 con 0x7ffb801097b0 2024-02-14T17:43:20.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.807+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7ffb74088890 con 0x7ffb801097b0 2024-02-14T17:43:20.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:20.811+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7ffb74055c70 con 0x7ffb801097b0 2024-02-14T17:43:21.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:21.005+0000 7ffb8548b700 1 -- 172.21.15.178:0/3236966946 --> v1:172.21.15.178:6800/4205070280 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb6c000bc0 con 0x7ffb6806e9f0 2024-02-14T17:43:21.007 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:20 smithi178 ceph-mon[30321]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:21.008 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:20 smithi178 ceph-mon[39811]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:21.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:21.017+0000 7ffb67fff700 1 -- 172.21.15.178:0/3236966946 <== mgr.14595 v1:172.21.15.178:6800/4205070280 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+383696 (unknown 2965378022 0 2410199087) 0x7ffb6c000bc0 con 0x7ffb6806e9f0 2024-02-14T17:43:21.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:43:21.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:21.020+0000 7ffb8548b700 1 -- 172.21.15.178:0/3236966946 >> v1:172.21.15.178:6800/4205070280 conn(0x7ffb6806e9f0 legacy=0x7ffb68070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:21.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:21.020+0000 7ffb8548b700 1 -- 172.21.15.178:0/3236966946 >> v1:172.21.15.178:6790/0 conn(0x7ffb801097b0 legacy=0x7ffb801a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:21.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:21.020+0000 7ffb8548b700 1 -- 172.21.15.178:0/3236966946 shutdown_connections 2024-02-14T17:43:21.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:21.020+0000 7ffb8548b700 1 -- 172.21.15.178:0/3236966946 >> 172.21.15.178:0/3236966946 conn(0x7ffb800fc410 msgr2=0x7ffb801105c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:21.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:21.020+0000 7ffb8548b700 1 -- 172.21.15.178:0/3236966946 shutdown_connections 2024-02-14T17:43:21.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:21.020+0000 7ffb8548b700 1 -- 172.21.15.178:0/3236966946 wait complete. 2024-02-14T17:43:21.030 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-02-14T17:43:21.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:20 smithi184 ceph-mon[33536]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:21.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:43:20 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:43:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:21 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:22.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:21 smithi184 ceph-mon[33536]: from='client.34637 v1:172.21.15.178:0/3236966946' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:43:22.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:21 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:21 smithi178 ceph-mon[39811]: from='client.34637 v1:172.21.15.178:0/3236966946' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:43:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:21 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:21 smithi178 ceph-mon[30321]: from='client.34637 v1:172.21.15.178:0/3236966946' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:43:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:22 smithi184 ceph-mon[33536]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:23.220 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:22 smithi178 ceph-mon[30321]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:23.220 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:22 smithi178 ceph-mon[39811]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:24.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:24 smithi178 systemd[1]: Starting Ceph alertmanager.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:43:24.924 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-02-14T17:43:19.318864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1004,"num_read_kb":850,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":1771544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":248624,"kb_used_data":7516,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":749712592,"statfs":{"total":767960285184,"available":767705694208,"internally_reserved":0,"allocated":7696384,"data_stored":4079964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":246795830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002367"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353263+0000","last_change":"2024-02-14T17:42:04.072035+0000","last_active":"2024-02-14T17:42:39.353263+0000","last_peered":"2024-02-14T17:42:39.353263+0000","last_clean":"2024-02-14T17:42:39.353263+0000","last_became_active":"2024-02-14T17:42:04.071721+0000","last_became_peered":"2024-02-14T17:42:04.071721+0000","last_unstale":"2024-02-14T17:42:39.353263+0000","last_undegraded":"2024-02-14T17:42:39.353263+0000","last_fullsized":"2024-02-14T17:42:39.353263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:42:02.683826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354362+0000","last_change":"2024-02-14T17:42:02.068619+0000","last_active":"2024-02-14T17:42:39.354362+0000","last_peered":"2024-02-14T17:42:39.354362+0000","last_clean":"2024-02-14T17:42:39.354362+0000","last_became_active":"2024-02-14T17:42:02.068319+0000","last_became_peered":"2024-02-14T17:42:02.068319+0000","last_unstale":"2024-02-14T17:42:39.354362+0000","last_undegraded":"2024-02-14T17:42:39.354362+0000","last_fullsized":"2024-02-14T17:42:39.354362+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:15:44.584475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302130+0000","last_change":"2024-02-14T17:41:58.156367+0000","last_active":"2024-02-14T17:42:39.302130+0000","last_peered":"2024-02-14T17:42:39.302130+0000","last_clean":"2024-02-14T17:42:39.302130+0000","last_became_active":"2024-02-14T17:41:58.155580+0000","last_became_peered":"2024-02-14T17:41:58.155580+0000","last_unstale":"2024-02-14T17:42:39.302130+0000","last_undegraded":"2024-02-14T17:42:39.302130+0000","last_fullsized":"2024-02-14T17:42:39.302130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:23:00.873224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"70'9","reported_seq":61,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353248+0000","last_change":"2024-02-14T17:42:22.157602+0000","last_active":"2024-02-14T17:42:39.353248+0000","last_peered":"2024-02-14T17:42:39.353248+0000","last_clean":"2024-02-14T17:42:39.353248+0000","last_became_active":"2024-02-14T17:42:22.150824+0000","last_became_peered":"2024-02-14T17:42:22.150824+0000","last_unstale":"2024-02-14T17:42:39.353248+0000","last_undegraded":"2024-02-14T17:42:39.353248+0000","last_fullsized":"2024-02-14T17:42:39.353248+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:42:15.412332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355196+0000","last_change":"2024-02-14T17:42:04.356662+0000","last_active":"2024-02-14T17:42:39.355196+0000","last_peered":"2024-02-14T17:42:39.355196+0000","last_clean":"2024-02-14T17:42:39.355196+0000","last_became_active":"2024-02-14T17:42:04.355722+0000","last_became_peered":"2024-02-14T17:42:04.355722+0000","last_unstale":"2024-02-14T17:42:39.355196+0000","last_undegraded":"2024-02-14T17:42:39.355196+0000","last_fullsized":"2024-02-14T17:42:39.355196+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:48:51.796142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353383+0000","last_change":"2024-02-14T17:42:02.072928+0000","last_active":"2024-02-14T17:42:39.353383+0000","last_peered":"2024-02-14T17:42:39.353383+0000","last_clean":"2024-02-14T17:42:39.353383+0000","last_became_active":"2024-02-14T17:42:02.072647+0000","last_became_peered":"2024-02-14T17:42:02.072647+0000","last_unstale":"2024-02-14T17:42:39.353383+0000","last_undegraded":"2024-02-14T17:42:39.353383+0000","last_fullsized":"2024-02-14T17:42:39.353383+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:25:03.883605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355063+0000","last_change":"2024-02-14T17:41:58.155858+0000","last_active":"2024-02-14T17:42:39.355063+0000","last_peered":"2024-02-14T17:42:39.355063+0000","last_clean":"2024-02-14T17:42:39.355063+0000","last_became_active":"2024-02-14T17:41:58.155562+0000","last_became_peered":"2024-02-14T17:41:58.155562+0000","last_unstale":"2024-02-14T17:42:39.355063+0000","last_undegraded":"2024-02-14T17:42:39.355063+0000","last_fullsized":"2024-02-14T17:42:39.355063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:31:59.146624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354423+0000","last_change":"2024-02-14T17:42:00.052945+0000","last_active":"2024-02-14T17:42:39.354423+0000","last_peered":"2024-02-14T17:42:39.354423+0000","last_clean":"2024-02-14T17:42:39.354423+0000","last_became_active":"2024-02-14T17:42:00.052565+0000","last_became_peered":"2024-02-14T17:42:00.052565+0000","last_unstale":"2024-02-14T17:42:39.354423+0000","last_undegraded":"2024-02-14T17:42:39.354423+0000","last_fullsized":"2024-02-14T17:42:39.354423+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:55:28.405525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.301610+0000","last_change":"2024-02-14T17:42:04.070998+0000","last_active":"2024-02-14T17:42:39.301610+0000","last_peered":"2024-02-14T17:42:39.301610+0000","last_clean":"2024-02-14T17:42:39.301610+0000","last_became_active":"2024-02-14T17:42:04.070678+0000","last_became_peered":"2024-02-14T17:42:04.070678+0000","last_unstale":"2024-02-14T17:42:39.301610+0000","last_undegraded":"2024-02-14T17:42:39.301610+0000","last_fullsized":"2024-02-14T17:42:39.301610+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:12:56.679400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353411+0000","last_change":"2024-02-14T17:42:02.067757+0000","last_active":"2024-02-14T17:42:39.353411+0000","last_peered":"2024-02-14T17:42:39.353411+0000","last_clean":"2024-02-14T17:42:39.353411+0000","last_became_active":"2024-02-14T17:42:02.067110+0000","last_became_peered":"2024-02-14T17:42:02.067110+0000","last_unstale":"2024-02-14T17:42:39.353411+0000","last_undegraded":"2024-02-14T17:42:39.353411+0000","last_fullsized":"2024-02-14T17:42:39.353411+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:58:19.455017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353857+0000","last_change":"2024-02-14T17:41:58.042092+0000","last_active":"2024-02-14T17:42:39.353857+0000","last_peered":"2024-02-14T17:42:39.353857+0000","last_clean":"2024-02-14T17:42:39.353857+0000","last_became_active":"2024-02-14T17:41:58.041787+0000","last_became_peered":"2024-02-14T17:41:58.041787+0000","last_unstale":"2024-02-14T17:42:39.353857+0000","last_undegraded":"2024-02-14T17:42:39.353857+0000","last_fullsized":"2024-02-14T17:42:39.353857+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:58:43.706000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353533+0000","last_change":"2024-02-14T17:42:00.050014+0000","last_active":"2024-02-14T17:42:39.353533+0000","last_peered":"2024-02-14T17:42:39.353533+0000","last_clean":"2024-02-14T17:42:39.353533+0000","last_became_active":"2024-02-14T17:42:00.049653+0000","last_became_peered":"2024-02-14T17:42:00.049653+0000","last_unstale":"2024-02-14T17:42:39.353533+0000","last_undegraded":"2024-02-14T17:42:39.353533+0000","last_fullsized":"2024-02-14T17:42:39.353533+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:25.308096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353185+0000","last_change":"2024-02-14T17:42:04.075850+0000","last_active":"2024-02-14T17:42:39.353185+0000","last_peered":"2024-02-14T17:42:39.353185+0000","last_clean":"2024-02-14T17:42:39.353185+0000","last_became_active":"2024-02-14T17:42:04.075600+0000","last_became_peered":"2024-02-14T17:42:04.075600+0000","last_unstale":"2024-02-14T17:42:39.353185+0000","last_undegraded":"2024-02-14T17:42:39.353185+0000","last_fullsized":"2024-02-14T17:42:39.353185+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:36:11.838782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304196+0000","last_change":"2024-02-14T17:42:02.073943+0000","last_active":"2024-02-14T17:42:39.304196+0000","last_peered":"2024-02-14T17:42:39.304196+0000","last_clean":"2024-02-14T17:42:39.304196+0000","last_became_active":"2024-02-14T17:42:02.073635+0000","last_became_peered":"2024-02-14T17:42:02.073635+0000","last_unstale":"2024-02-14T17:42:39.304196+0000","last_undegraded":"2024-02-14T17:42:39.304196+0000","last_fullsized":"2024-02-14T17:42:39.304196+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:55:30.133503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353728+0000","last_change":"2024-02-14T17:41:58.042442+0000","last_active":"2024-02-14T17:42:39.353728+0000","last_peered":"2024-02-14T17:42:39.353728+0000","last_clean":"2024-02-14T17:42:39.353728+0000","last_became_active":"2024-02-14T17:41:58.042026+0000","last_became_peered":"2024-02-14T17:41:58.042026+0000","last_unstale":"2024-02-14T17:42:39.353728+0000","last_undegraded":"2024-02-14T17:42:39.353728+0000","last_fullsized":"2024-02-14T17:42:39.353728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:38:24.035282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"70'5","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354021+0000","last_change":"2024-02-14T17:42:00.050443+0000","last_active":"2024-02-14T17:42:39.354021+0000","last_peered":"2024-02-14T17:42:39.354021+0000","last_clean":"2024-02-14T17:42:39.354021+0000","last_became_active":"2024-02-14T17:42:00.049854+0000","last_became_peered":"2024-02-14T17:42:00.049854+0000","last_unstale":"2024-02-14T17:42:39.354021+0000","last_undegraded":"2024-02-14T17:42:39.354021+0000","last_fullsized":"2024-02-14T17:42:39.354021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:12:17.646365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354583+0000","last_change":"2024-02-14T17:42:04.355188+0000","last_active":"2024-02-14T17:42:39.354583+0000","last_peered":"2024-02-14T17:42:39.354583+0000","last_clean":"2024-02-14T17:42:39.354583+0000","last_became_active":"2024-02-14T17:42:04.354880+0000","last_became_peered":"2024-02-14T17:42:04.354880+0000","last_unstale":"2024-02-14T17:42:39.354583+0000","last_undegraded":"2024-02-14T17:42:39.354583+0000","last_fullsized":"2024-02-14T17:42:39.354583+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:51:09.492406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355079+0000","last_change":"2024-02-14T17:42:02.073499+0000","last_active":"2024-02-14T17:42:39.355079+0000","last_peered":"2024-02-14T17:42:39.355079+0000","last_clean":"2024-02-14T17:42:39.355079+0000","last_became_active":"2024-02-14T17:42:02.073009+0000","last_became_peered":"2024-02-14T17:42:02.073009+0000","last_unstale":"2024-02-14T17:42:39.355079+0000","last_undegraded":"2024-02-14T17:42:39.355079+0000","last_fullsized":"2024-02-14T17:42:39.355079+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:01:34.577460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302092+0000","last_change":"2024-02-14T17:41:58.032401+0000","last_active":"2024-02-14T17:42:39.302092+0000","last_peered":"2024-02-14T17:42:39.302092+0000","last_clean":"2024-02-14T17:42:39.302092+0000","last_became_active":"2024-02-14T17:41:58.032122+0000","last_became_peered":"2024-02-14T17:41:58.032122+0000","last_unstale":"2024-02-14T17:42:39.302092+0000","last_undegraded":"2024-02-14T17:42:39.302092+0000","last_fullsized":"2024-02-14T17:42:39.302092+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:53:00.320513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353791+0000","last_change":"2024-02-14T17:42:00.052767+0000","last_active":"2024-02-14T17:42:39.353791+0000","last_peered":"2024-02-14T17:42:39.353791+0000","last_clean":"2024-02-14T17:42:39.353791+0000","last_became_active":"2024-02-14T17:42:00.052424+0000","last_became_peered":"2024-02-14T17:42:00.052424+0000","last_unstale":"2024-02-14T17:42:39.353791+0000","last_undegraded":"2024-02-14T17:42:39.353791+0000","last_fullsized":"2024-02-14T17:42:39.353791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:53:01.074045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354796+0000","last_change":"2024-02-14T17:42:04.356240+0000","last_active":"2024-02-14T17:42:39.354796+0000","last_peered":"2024-02-14T17:42:39.354796+0000","last_clean":"2024-02-14T17:42:39.354796+0000","last_became_active":"2024-02-14T17:42:04.355591+0000","last_became_peered":"2024-02-14T17:42:04.355591+0000","last_unstale":"2024-02-14T17:42:39.354796+0000","last_undegraded":"2024-02-14T17:42:39.354796+0000","last_fullsized":"2024-02-14T17:42:39.354796+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:38:49.229772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354858+0000","last_change":"2024-02-14T17:42:02.067639+0000","last_active":"2024-02-14T17:42:39.354858+0000","last_peered":"2024-02-14T17:42:39.354858+0000","last_clean":"2024-02-14T17:42:39.354858+0000","last_became_active":"2024-02-14T17:42:02.067354+0000","last_became_peered":"2024-02-14T17:42:02.067354+0000","last_unstale":"2024-02-14T17:42:39.354858+0000","last_undegraded":"2024-02-14T17:42:39.354858+0000","last_fullsized":"2024-02-14T17:42:39.354858+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:42:45.684379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353659+0000","last_change":"2024-02-14T17:41:58.156267+0000","last_active":"2024-02-14T17:42:39.353659+0000","last_peered":"2024-02-14T17:42:39.353659+0000","last_clean":"2024-02-14T17:42:39.353659+0000","last_became_active":"2024-02-14T17:41:58.155909+0000","last_became_peered":"2024-02-14T17:41:58.155909+0000","last_unstale":"2024-02-14T17:42:39.353659+0000","last_undegraded":"2024-02-14T17:42:39.353659+0000","last_fullsized":"2024-02-14T17:42:39.353659+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:17:21.006331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"70'12","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354614+0000","last_change":"2024-02-14T17:42:21.971331+0000","last_active":"2024-02-14T17:42:39.354614+0000","last_peered":"2024-02-14T17:42:39.354614+0000","last_clean":"2024-02-14T17:42:39.354614+0000","last_became_active":"2024-02-14T17:42:21.965142+0000","last_became_peered":"2024-02-14T17:42:21.965142+0000","last_unstale":"2024-02-14T17:42:39.354614+0000","last_undegraded":"2024-02-14T17:42:39.354614+0000","last_fullsized":"2024-02-14T17:42:39.354614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:53:27.259376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305791+0000","last_change":"2024-02-14T17:42:04.074529+0000","last_active":"2024-02-14T17:42:39.305791+0000","last_peered":"2024-02-14T17:42:39.305791+0000","last_clean":"2024-02-14T17:42:39.305791+0000","last_became_active":"2024-02-14T17:42:04.074140+0000","last_became_peered":"2024-02-14T17:42:04.074140+0000","last_unstale":"2024-02-14T17:42:39.305791+0000","last_undegraded":"2024-02-14T17:42:39.305791+0000","last_fullsized":"2024-02-14T17:42:39.305791+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:31:51.240899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"70'8","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:08.208892+0000","last_change":"2024-02-14T17:42:02.061550+0000","last_active":"2024-02-14T17:43:08.208892+0000","last_peered":"2024-02-14T17:43:08.208892+0000","last_clean":"2024-02-14T17:43:08.208892+0000","last_became_active":"2024-02-14T17:42:02.061241+0000","last_became_peered":"2024-02-14T17:42:02.061241+0000","last_unstale":"2024-02-14T17:43:08.208892+0000","last_undegraded":"2024-02-14T17:43:08.208892+0000","last_fullsized":"2024-02-14T17:43:08.208892+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:25:57.686521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"63'2","reported_seq":40,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354444+0000","last_change":"2024-02-14T17:41:58.033635+0000","last_active":"2024-02-14T17:42:39.354444+0000","last_peered":"2024-02-14T17:42:39.354444+0000","last_clean":"2024-02-14T17:42:39.354444+0000","last_became_active":"2024-02-14T17:41:58.033312+0000","last_became_peered":"2024-02-14T17:41:58.033312+0000","last_unstale":"2024-02-14T17:42:39.354444+0000","last_undegraded":"2024-02-14T17:42:39.354444+0000","last_fullsized":"2024-02-14T17:42:39.354444+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:27:03.220076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302598+0000","last_change":"2024-02-14T17:42:00.139996+0000","last_active":"2024-02-14T17:42:39.302598+0000","last_peered":"2024-02-14T17:42:39.302598+0000","last_clean":"2024-02-14T17:42:39.302598+0000","last_became_active":"2024-02-14T17:42:00.139681+0000","last_became_peered":"2024-02-14T17:42:00.139681+0000","last_unstale":"2024-02-14T17:42:39.302598+0000","last_undegraded":"2024-02-14T17:42:39.302598+0000","last_fullsized":"2024-02-14T17:42:39.302598+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:59:44.349762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354789+0000","last_change":"2024-02-14T17:42:04.071580+0000","last_active":"2024-02-14T17:42:39.354789+0000","last_peered":"2024-02-14T17:42:39.354789+0000","last_clean":"2024-02-14T17:42:39.354789+0000","last_became_active":"2024-02-14T17:42:04.071220+0000","last_became_peered":"2024-02-14T17:42:04.071220+0000","last_unstale":"2024-02-14T17:42:39.354789+0000","last_undegraded":"2024-02-14T17:42:39.354789+0000","last_fullsized":"2024-02-14T17:42:39.354789+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:21:07.253754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302936+0000","last_change":"2024-02-14T17:42:02.061685+0000","last_active":"2024-02-14T17:42:39.302936+0000","last_peered":"2024-02-14T17:42:39.302936+0000","last_clean":"2024-02-14T17:42:39.302936+0000","last_became_active":"2024-02-14T17:42:02.061361+0000","last_became_peered":"2024-02-14T17:42:02.061361+0000","last_unstale":"2024-02-14T17:42:39.302936+0000","last_undegraded":"2024-02-14T17:42:39.302936+0000","last_fullsized":"2024-02-14T17:42:39.302936+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:56:26.735679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353275+0000","last_change":"2024-02-14T17:41:58.031978+0000","last_active":"2024-02-14T17:42:39.353275+0000","last_peered":"2024-02-14T17:42:39.353275+0000","last_clean":"2024-02-14T17:42:39.353275+0000","last_became_active":"2024-02-14T17:41:58.031514+0000","last_became_peered":"2024-02-14T17:41:58.031514+0000","last_unstale":"2024-02-14T17:42:39.353275+0000","last_undegraded":"2024-02-14T17:42:39.353275+0000","last_fullsized":"2024-02-14T17:42:39.353275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:42:22.060395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303881+0000","last_change":"2024-02-14T17:42:00.053869+0000","last_active":"2024-02-14T17:42:39.303881+0000","last_peered":"2024-02-14T17:42:39.303881+0000","last_clean":"2024-02-14T17:42:39.303881+0000","last_became_active":"2024-02-14T17:42:00.053411+0000","last_became_peered":"2024-02-14T17:42:00.053411+0000","last_unstale":"2024-02-14T17:42:39.303881+0000","last_undegraded":"2024-02-14T17:42:39.303881+0000","last_fullsized":"2024-02-14T17:42:39.303881+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:37:56.338543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304144+0000","last_change":"2024-02-14T17:42:04.074218+0000","last_active":"2024-02-14T17:42:39.304144+0000","last_peered":"2024-02-14T17:42:39.304144+0000","last_clean":"2024-02-14T17:42:39.304144+0000","last_became_active":"2024-02-14T17:42:04.073887+0000","last_became_peered":"2024-02-14T17:42:04.073887+0000","last_unstale":"2024-02-14T17:42:39.304144+0000","last_undegraded":"2024-02-14T17:42:39.304144+0000","last_fullsized":"2024-02-14T17:42:39.304144+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:24:10.551620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302750+0000","last_change":"2024-02-14T17:42:02.063328+0000","last_active":"2024-02-14T17:42:39.302750+0000","last_peered":"2024-02-14T17:42:39.302750+0000","last_clean":"2024-02-14T17:42:39.302750+0000","last_became_active":"2024-02-14T17:42:02.063028+0000","last_became_peered":"2024-02-14T17:42:02.063028+0000","last_unstale":"2024-02-14T17:42:39.302750+0000","last_undegraded":"2024-02-14T17:42:39.302750+0000","last_fullsized":"2024-02-14T17:42:39.302750+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:50:20.781895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302026+0000","last_change":"2024-02-14T17:41:58.039088+0000","last_active":"2024-02-14T17:42:39.302026+0000","last_peered":"2024-02-14T17:42:39.302026+0000","last_clean":"2024-02-14T17:42:39.302026+0000","last_became_active":"2024-02-14T17:41:58.037943+0000","last_became_peered":"2024-02-14T17:41:58.037943+0000","last_unstale":"2024-02-14T17:42:39.302026+0000","last_undegraded":"2024-02-14T17:42:39.302026+0000","last_fullsized":"2024-02-14T17:42:39.302026+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:01:00.146322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"70'4","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353382+0000","last_change":"2024-02-14T17:42:00.054135+0000","last_active":"2024-02-14T17:42:39.353382+0000","last_peered":"2024-02-14T17:42:39.353382+0000","last_clean":"2024-02-14T17:42:39.353382+0000","last_became_active":"2024-02-14T17:42:00.053812+0000","last_became_peered":"2024-02-14T17:42:00.053812+0000","last_unstale":"2024-02-14T17:42:39.353382+0000","last_undegraded":"2024-02-14T17:42:39.353382+0000","last_fullsized":"2024-02-14T17:42:39.353382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:31:24.841120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354356+0000","last_change":"2024-02-14T17:42:04.355940+0000","last_active":"2024-02-14T17:42:39.354356+0000","last_peered":"2024-02-14T17:42:39.354356+0000","last_clean":"2024-02-14T17:42:39.354356+0000","last_became_active":"2024-02-14T17:42:04.355168+0000","last_became_peered":"2024-02-14T17:42:04.355168+0000","last_unstale":"2024-02-14T17:42:39.354356+0000","last_undegraded":"2024-02-14T17:42:39.354356+0000","last_fullsized":"2024-02-14T17:42:39.354356+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:24:32.385496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303778+0000","last_change":"2024-02-14T17:42:02.064688+0000","last_active":"2024-02-14T17:42:39.303778+0000","last_peered":"2024-02-14T17:42:39.303778+0000","last_clean":"2024-02-14T17:42:39.303778+0000","last_became_active":"2024-02-14T17:42:02.064392+0000","last_became_peered":"2024-02-14T17:42:02.064392+0000","last_unstale":"2024-02-14T17:42:39.303778+0000","last_undegraded":"2024-02-14T17:42:39.303778+0000","last_fullsized":"2024-02-14T17:42:39.303778+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:02:12.496777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354227+0000","last_change":"2024-02-14T17:41:58.043010+0000","last_active":"2024-02-14T17:42:39.354227+0000","last_peered":"2024-02-14T17:42:39.354227+0000","last_clean":"2024-02-14T17:42:39.354227+0000","last_became_active":"2024-02-14T17:41:58.042656+0000","last_became_peered":"2024-02-14T17:41:58.042656+0000","last_unstale":"2024-02-14T17:42:39.354227+0000","last_undegraded":"2024-02-14T17:42:39.354227+0000","last_fullsized":"2024-02-14T17:42:39.354227+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:00:59.476277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354287+0000","last_change":"2024-02-14T17:42:00.141163+0000","last_active":"2024-02-14T17:42:39.354287+0000","last_peered":"2024-02-14T17:42:39.354287+0000","last_clean":"2024-02-14T17:42:39.354287+0000","last_became_active":"2024-02-14T17:42:00.140851+0000","last_became_peered":"2024-02-14T17:42:00.140851+0000","last_unstale":"2024-02-14T17:42:39.354287+0000","last_undegraded":"2024-02-14T17:42:39.354287+0000","last_fullsized":"2024-02-14T17:42:39.354287+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:03:15.318523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.300969+0000","last_change":"2024-02-14T17:42:04.065732+0000","last_active":"2024-02-14T17:42:39.300969+0000","last_peered":"2024-02-14T17:42:39.300969+0000","last_clean":"2024-02-14T17:42:39.300969+0000","last_became_active":"2024-02-14T17:42:04.065376+0000","last_became_peered":"2024-02-14T17:42:04.065376+0000","last_unstale":"2024-02-14T17:42:39.300969+0000","last_undegraded":"2024-02-14T17:42:39.300969+0000","last_fullsized":"2024-02-14T17:42:39.300969+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:06:47.477195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355196+0000","last_change":"2024-02-14T17:42:02.068711+0000","last_active":"2024-02-14T17:42:39.355196+0000","last_peered":"2024-02-14T17:42:39.355196+0000","last_clean":"2024-02-14T17:42:39.355196+0000","last_became_active":"2024-02-14T17:42:02.068404+0000","last_became_peered":"2024-02-14T17:42:02.068404+0000","last_unstale":"2024-02-14T17:42:39.355196+0000","last_undegraded":"2024-02-14T17:42:39.355196+0000","last_fullsized":"2024-02-14T17:42:39.355196+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:53:52.215133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304866+0000","last_change":"2024-02-14T17:41:58.036293+0000","last_active":"2024-02-14T17:42:39.304866+0000","last_peered":"2024-02-14T17:42:39.304866+0000","last_clean":"2024-02-14T17:42:39.304866+0000","last_became_active":"2024-02-14T17:41:58.035951+0000","last_became_peered":"2024-02-14T17:41:58.035951+0000","last_unstale":"2024-02-14T17:42:39.304866+0000","last_undegraded":"2024-02-14T17:42:39.304866+0000","last_fullsized":"2024-02-14T17:42:39.304866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:18:52.020383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354340+0000","last_change":"2024-02-14T17:42:00.047243+0000","last_active":"2024-02-14T17:42:39.354340+0000","last_peered":"2024-02-14T17:42:39.354340+0000","last_clean":"2024-02-14T17:42:39.354340+0000","last_became_active":"2024-02-14T17:42:00.046438+0000","last_became_peered":"2024-02-14T17:42:00.046438+0000","last_unstale":"2024-02-14T17:42:39.354340+0000","last_undegraded":"2024-02-14T17:42:39.354340+0000","last_fullsized":"2024-02-14T17:42:39.354340+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:25:28.078296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353807+0000","last_change":"2024-02-14T17:42:04.080840+0000","last_active":"2024-02-14T17:42:39.353807+0000","last_peered":"2024-02-14T17:42:39.353807+0000","last_clean":"2024-02-14T17:42:39.353807+0000","last_became_active":"2024-02-14T17:42:04.080456+0000","last_became_peered":"2024-02-14T17:42:04.080456+0000","last_unstale":"2024-02-14T17:42:39.353807+0000","last_undegraded":"2024-02-14T17:42:39.353807+0000","last_fullsized":"2024-02-14T17:42:39.353807+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:50:43.817082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354126+0000","last_change":"2024-02-14T17:42:02.071624+0000","last_active":"2024-02-14T17:42:39.354126+0000","last_peered":"2024-02-14T17:42:39.354126+0000","last_clean":"2024-02-14T17:42:39.354126+0000","last_became_active":"2024-02-14T17:42:02.071279+0000","last_became_peered":"2024-02-14T17:42:02.071279+0000","last_unstale":"2024-02-14T17:42:39.354126+0000","last_undegraded":"2024-02-14T17:42:39.354126+0000","last_fullsized":"2024-02-14T17:42:39.354126+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:50:59.905322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354731+0000","last_change":"2024-02-14T17:41:58.039692+0000","last_active":"2024-02-14T17:42:39.354731+0000","last_peered":"2024-02-14T17:42:39.354731+0000","last_clean":"2024-02-14T17:42:39.354731+0000","last_became_active":"2024-02-14T17:41:58.039373+0000","last_became_peered":"2024-02-14T17:41:58.039373+0000","last_unstale":"2024-02-14T17:42:39.354731+0000","last_undegraded":"2024-02-14T17:42:39.354731+0000","last_fullsized":"2024-02-14T17:42:39.354731+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:18:05.014382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354096+0000","last_change":"2024-02-14T17:42:00.043896+0000","last_active":"2024-02-14T17:42:39.354096+0000","last_peered":"2024-02-14T17:42:39.354096+0000","last_clean":"2024-02-14T17:42:39.354096+0000","last_became_active":"2024-02-14T17:42:00.043505+0000","last_became_peered":"2024-02-14T17:42:00.043505+0000","last_unstale":"2024-02-14T17:42:39.354096+0000","last_undegraded":"2024-02-14T17:42:39.354096+0000","last_fullsized":"2024-02-14T17:42:39.354096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:26:03.458942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302248+0000","last_change":"2024-02-14T17:42:04.075674+0000","last_active":"2024-02-14T17:42:39.302248+0000","last_peered":"2024-02-14T17:42:39.302248+0000","last_clean":"2024-02-14T17:42:39.302248+0000","last_became_active":"2024-02-14T17:42:04.075391+0000","last_became_peered":"2024-02-14T17:42:04.075391+0000","last_unstale":"2024-02-14T17:42:39.302248+0000","last_undegraded":"2024-02-14T17:42:39.302248+0000","last_fullsized":"2024-02-14T17:42:39.302248+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:47:51.331635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"70'8","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:08.208960+0000","last_change":"2024-02-14T17:42:02.065603+0000","last_active":"2024-02-14T17:43:08.208960+0000","last_peered":"2024-02-14T17:43:08.208960+0000","last_clean":"2024-02-14T17:43:08.208960+0000","last_became_active":"2024-02-14T17:42:02.065271+0000","last_became_peered":"2024-02-14T17:42:02.065271+0000","last_unstale":"2024-02-14T17:43:08.208960+0000","last_undegraded":"2024-02-14T17:43:08.208960+0000","last_fullsized":"2024-02-14T17:43:08.208960+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:00:47.566817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304316+0000","last_change":"2024-02-14T17:41:58.157507+0000","last_active":"2024-02-14T17:42:39.304316+0000","last_peered":"2024-02-14T17:42:39.304316+0000","last_clean":"2024-02-14T17:42:39.304316+0000","last_became_active":"2024-02-14T17:41:58.157133+0000","last_became_peered":"2024-02-14T17:41:58.157133+0000","last_unstale":"2024-02-14T17:42:39.304316+0000","last_undegraded":"2024-02-14T17:42:39.304316+0000","last_fullsized":"2024-02-14T17:42:39.304316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:21:37.614580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354828+0000","last_change":"2024-02-14T17:42:00.139951+0000","last_active":"2024-02-14T17:42:39.354828+0000","last_peered":"2024-02-14T17:42:39.354828+0000","last_clean":"2024-02-14T17:42:39.354828+0000","last_became_active":"2024-02-14T17:42:00.139630+0000","last_became_peered":"2024-02-14T17:42:00.139630+0000","last_unstale":"2024-02-14T17:42:39.354828+0000","last_undegraded":"2024-02-14T17:42:39.354828+0000","last_fullsized":"2024-02-14T17:42:39.354828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:20:19.278922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354548+0000","last_change":"2024-02-14T17:42:04.074089+0000","last_active":"2024-02-14T17:42:39.354548+0000","last_peered":"2024-02-14T17:42:39.354548+0000","last_clean":"2024-02-14T17:42:39.354548+0000","last_became_active":"2024-02-14T17:42:04.073825+0000","last_became_peered":"2024-02-14T17:42:04.073825+0000","last_unstale":"2024-02-14T17:42:39.354548+0000","last_undegraded":"2024-02-14T17:42:39.354548+0000","last_fullsized":"2024-02-14T17:42:39.354548+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:51.672907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:08.208923+0000","last_change":"2024-02-14T17:42:02.070809+0000","last_active":"2024-02-14T17:43:08.208923+0000","last_peered":"2024-02-14T17:43:08.208923+0000","last_clean":"2024-02-14T17:43:08.208923+0000","last_became_active":"2024-02-14T17:42:02.070479+0000","last_became_peered":"2024-02-14T17:42:02.070479+0000","last_unstale":"2024-02-14T17:43:08.208923+0000","last_undegraded":"2024-02-14T17:43:08.208923+0000","last_fullsized":"2024-02-14T17:43:08.208923+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:24:01.484179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.301209+0000","last_change":"2024-02-14T17:41:58.031279+0000","last_active":"2024-02-14T17:42:39.301209+0000","last_peered":"2024-02-14T17:42:39.301209+0000","last_clean":"2024-02-14T17:42:39.301209+0000","last_became_active":"2024-02-14T17:41:58.030713+0000","last_became_peered":"2024-02-14T17:41:58.030713+0000","last_unstale":"2024-02-14T17:42:39.301209+0000","last_undegraded":"2024-02-14T17:42:39.301209+0000","last_fullsized":"2024-02-14T17:42:39.301209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:49:54.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354061+0000","last_change":"2024-02-14T17:42:00.140621+0000","last_active":"2024-02-14T17:42:39.354061+0000","last_peered":"2024-02-14T17:42:39.354061+0000","last_clean":"2024-02-14T17:42:39.354061+0000","last_became_active":"2024-02-14T17:42:00.140301+0000","last_became_peered":"2024-02-14T17:42:00.140301+0000","last_unstale":"2024-02-14T17:42:39.354061+0000","last_undegraded":"2024-02-14T17:42:39.354061+0000","last_fullsized":"2024-02-14T17:42:39.354061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:52:11.686584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354598+0000","last_change":"2024-02-14T17:42:04.077953+0000","last_active":"2024-02-14T17:42:39.354598+0000","last_peered":"2024-02-14T17:42:39.354598+0000","last_clean":"2024-02-14T17:42:39.354598+0000","last_became_active":"2024-02-14T17:42:04.077620+0000","last_became_peered":"2024-02-14T17:42:04.077620+0000","last_unstale":"2024-02-14T17:42:39.354598+0000","last_undegraded":"2024-02-14T17:42:39.354598+0000","last_fullsized":"2024-02-14T17:42:39.354598+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:02:01.105691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355026+0000","last_change":"2024-02-14T17:42:02.069810+0000","last_active":"2024-02-14T17:42:39.355026+0000","last_peered":"2024-02-14T17:42:39.355026+0000","last_clean":"2024-02-14T17:42:39.355026+0000","last_became_active":"2024-02-14T17:42:02.069477+0000","last_became_peered":"2024-02-14T17:42:02.069477+0000","last_unstale":"2024-02-14T17:42:39.355026+0000","last_undegraded":"2024-02-14T17:42:39.355026+0000","last_fullsized":"2024-02-14T17:42:39.355026+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:20:15.353432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303936+0000","last_change":"2024-02-14T17:41:58.037798+0000","last_active":"2024-02-14T17:42:39.303936+0000","last_peered":"2024-02-14T17:42:39.303936+0000","last_clean":"2024-02-14T17:42:39.303936+0000","last_became_active":"2024-02-14T17:41:58.036822+0000","last_became_peered":"2024-02-14T17:41:58.036822+0000","last_unstale":"2024-02-14T17:42:39.303936+0000","last_undegraded":"2024-02-14T17:42:39.303936+0000","last_fullsized":"2024-02-14T17:42:39.303936+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:22:39.318435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302733+0000","last_change":"2024-02-14T17:42:00.140533+0000","last_active":"2024-02-14T17:42:39.302733+0000","last_peered":"2024-02-14T17:42:39.302733+0000","last_clean":"2024-02-14T17:42:39.302733+0000","last_became_active":"2024-02-14T17:42:00.139838+0000","last_became_peered":"2024-02-14T17:42:00.139838+0000","last_unstale":"2024-02-14T17:42:39.302733+0000","last_undegraded":"2024-02-14T17:42:39.302733+0000","last_fullsized":"2024-02-14T17:42:39.302733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:41:44.356387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304773+0000","last_change":"2024-02-14T17:42:04.357114+0000","last_active":"2024-02-14T17:42:39.304773+0000","last_peered":"2024-02-14T17:42:39.304773+0000","last_clean":"2024-02-14T17:42:39.304773+0000","last_became_active":"2024-02-14T17:42:04.356810+0000","last_became_peered":"2024-02-14T17:42:04.356810+0000","last_unstale":"2024-02-14T17:42:39.304773+0000","last_undegraded":"2024-02-14T17:42:39.304773+0000","last_fullsized":"2024-02-14T17:42:39.304773+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:27:17.103899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354231+0000","last_change":"2024-02-14T17:42:02.065716+0000","last_active":"2024-02-14T17:42:39.354231+0000","last_peered":"2024-02-14T17:42:39.354231+0000","last_clean":"2024-02-14T17:42:39.354231+0000","last_became_active":"2024-02-14T17:42:02.065386+0000","last_became_peered":"2024-02-14T17:42:02.065386+0000","last_unstale":"2024-02-14T17:42:39.354231+0000","last_undegraded":"2024-02-14T17:42:39.354231+0000","last_fullsized":"2024-02-14T17:42:39.354231+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:10:42.570899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302783+0000","last_change":"2024-02-14T17:41:58.040928+0000","last_active":"2024-02-14T17:42:39.302783+0000","last_peered":"2024-02-14T17:42:39.302783+0000","last_clean":"2024-02-14T17:42:39.302783+0000","last_became_active":"2024-02-14T17:41:58.040492+0000","last_became_peered":"2024-02-14T17:41:58.040492+0000","last_unstale":"2024-02-14T17:42:39.302783+0000","last_undegraded":"2024-02-14T17:42:39.302783+0000","last_fullsized":"2024-02-14T17:42:39.302783+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:29:52.954744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"70'6","reported_seq":38,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354973+0000","last_change":"2024-02-14T17:42:00.140217+0000","last_active":"2024-02-14T17:42:39.354973+0000","last_peered":"2024-02-14T17:42:39.354973+0000","last_clean":"2024-02-14T17:42:39.354973+0000","last_became_active":"2024-02-14T17:42:00.139585+0000","last_became_peered":"2024-02-14T17:42:00.139585+0000","last_unstale":"2024-02-14T17:42:39.354973+0000","last_undegraded":"2024-02-14T17:42:39.354973+0000","last_fullsized":"2024-02-14T17:42:39.354973+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:08:40.627653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353575+0000","last_change":"2024-02-14T17:42:04.356523+0000","last_active":"2024-02-14T17:42:39.353575+0000","last_peered":"2024-02-14T17:42:39.353575+0000","last_clean":"2024-02-14T17:42:39.353575+0000","last_became_active":"2024-02-14T17:42:04.356140+0000","last_became_peered":"2024-02-14T17:42:04.356140+0000","last_unstale":"2024-02-14T17:42:39.353575+0000","last_undegraded":"2024-02-14T17:42:39.353575+0000","last_fullsized":"2024-02-14T17:42:39.353575+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:17:03.561175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"63'2","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305392+0000","last_change":"2024-02-14T17:41:58.036370+0000","last_active":"2024-02-14T17:42:39.305392+0000","last_peered":"2024-02-14T17:42:39.305392+0000","last_clean":"2024-02-14T17:42:39.305392+0000","last_became_active":"2024-02-14T17:41:58.036059+0000","last_became_peered":"2024-02-14T17:41:58.036059+0000","last_unstale":"2024-02-14T17:42:39.305392+0000","last_undegraded":"2024-02-14T17:42:39.305392+0000","last_fullsized":"2024-02-14T17:42:39.305392+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:36:52.863582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353201+0000","last_change":"2024-02-14T17:42:00.140170+0000","last_active":"2024-02-14T17:42:39.353201+0000","last_peered":"2024-02-14T17:42:39.353201+0000","last_clean":"2024-02-14T17:42:39.353201+0000","last_became_active":"2024-02-14T17:42:00.139810+0000","last_became_peered":"2024-02-14T17:42:00.139810+0000","last_unstale":"2024-02-14T17:42:39.353201+0000","last_undegraded":"2024-02-14T17:42:39.353201+0000","last_fullsized":"2024-02-14T17:42:39.353201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:37:30.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:08.208922+0000","last_change":"2024-02-14T17:42:02.065804+0000","last_active":"2024-02-14T17:43:08.208922+0000","last_peered":"2024-02-14T17:43:08.208922+0000","last_clean":"2024-02-14T17:43:08.208922+0000","last_became_active":"2024-02-14T17:42:02.065425+0000","last_became_peered":"2024-02-14T17:42:02.065425+0000","last_unstale":"2024-02-14T17:43:08.208922+0000","last_undegraded":"2024-02-14T17:43:08.208922+0000","last_fullsized":"2024-02-14T17:43:08.208922+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:47:43.109365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354462+0000","last_change":"2024-02-14T17:42:04.078791+0000","last_active":"2024-02-14T17:42:39.354462+0000","last_peered":"2024-02-14T17:42:39.354462+0000","last_clean":"2024-02-14T17:42:39.354462+0000","last_became_active":"2024-02-14T17:42:04.078457+0000","last_became_peered":"2024-02-14T17:42:04.078457+0000","last_unstale":"2024-02-14T17:42:39.354462+0000","last_undegraded":"2024-02-14T17:42:39.354462+0000","last_fullsized":"2024-02-14T17:42:39.354462+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:24:46.172169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305441+0000","last_change":"2024-02-14T17:41:58.037428+0000","last_active":"2024-02-14T17:42:39.305441+0000","last_peered":"2024-02-14T17:42:39.305441+0000","last_clean":"2024-02-14T17:42:39.305441+0000","last_became_active":"2024-02-14T17:41:58.037084+0000","last_became_peered":"2024-02-14T17:41:58.037084+0000","last_unstale":"2024-02-14T17:42:39.305441+0000","last_undegraded":"2024-02-14T17:42:39.305441+0000","last_fullsized":"2024-02-14T17:42:39.305441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:53:18.073343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353783+0000","last_change":"2024-02-14T17:42:00.050716+0000","last_active":"2024-02-14T17:42:39.353783+0000","last_peered":"2024-02-14T17:42:39.353783+0000","last_clean":"2024-02-14T17:42:39.353783+0000","last_became_active":"2024-02-14T17:42:00.050000+0000","last_became_peered":"2024-02-14T17:42:00.050000+0000","last_unstale":"2024-02-14T17:42:39.353783+0000","last_undegraded":"2024-02-14T17:42:39.353783+0000","last_fullsized":"2024-02-14T17:42:39.353783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:50:10.957245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353000+0000","last_change":"2024-02-14T17:42:02.071692+0000","last_active":"2024-02-14T17:42:39.353000+0000","last_peered":"2024-02-14T17:42:39.353000+0000","last_clean":"2024-02-14T17:42:39.353000+0000","last_became_active":"2024-02-14T17:42:02.071380+0000","last_became_peered":"2024-02-14T17:42:02.071380+0000","last_unstale":"2024-02-14T17:42:39.353000+0000","last_undegraded":"2024-02-14T17:42:39.353000+0000","last_fullsized":"2024-02-14T17:42:39.353000+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:05:16.575354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303513+0000","last_change":"2024-02-14T17:42:04.080478+0000","last_active":"2024-02-14T17:42:39.303513+0000","last_peered":"2024-02-14T17:42:39.303513+0000","last_clean":"2024-02-14T17:42:39.303513+0000","last_became_active":"2024-02-14T17:42:04.080222+0000","last_became_peered":"2024-02-14T17:42:04.080222+0000","last_unstale":"2024-02-14T17:42:39.303513+0000","last_undegraded":"2024-02-14T17:42:39.303513+0000","last_fullsized":"2024-02-14T17:42:39.303513+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:40:27.894028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353576+0000","last_change":"2024-02-14T17:42:02.068426+0000","last_active":"2024-02-14T17:42:39.353576+0000","last_peered":"2024-02-14T17:42:39.353576+0000","last_clean":"2024-02-14T17:42:39.353576+0000","last_became_active":"2024-02-14T17:42:02.068068+0000","last_became_peered":"2024-02-14T17:42:02.068068+0000","last_unstale":"2024-02-14T17:42:39.353576+0000","last_undegraded":"2024-02-14T17:42:39.353576+0000","last_fullsized":"2024-02-14T17:42:39.353576+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:03:04.410879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305042+0000","last_change":"2024-02-14T17:41:58.040255+0000","last_active":"2024-02-14T17:42:39.305042+0000","last_peered":"2024-02-14T17:42:39.305042+0000","last_clean":"2024-02-14T17:42:39.305042+0000","last_became_active":"2024-02-14T17:41:58.039893+0000","last_became_peered":"2024-02-14T17:41:58.039893+0000","last_unstale":"2024-02-14T17:42:39.305042+0000","last_undegraded":"2024-02-14T17:42:39.305042+0000","last_fullsized":"2024-02-14T17:42:39.305042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:09:13.100236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"70'19","reported_seq":86,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303302+0000","last_change":"2024-02-14T17:42:21.979074+0000","last_active":"2024-02-14T17:42:39.303302+0000","last_peered":"2024-02-14T17:42:39.303302+0000","last_clean":"2024-02-14T17:42:39.303302+0000","last_became_active":"2024-02-14T17:42:21.965453+0000","last_became_peered":"2024-02-14T17:42:21.965453+0000","last_unstale":"2024-02-14T17:42:39.303302+0000","last_undegraded":"2024-02-14T17:42:39.303302+0000","last_fullsized":"2024-02-14T17:42:39.303302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:06:16.744933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.300965+0000","last_change":"2024-02-14T17:42:04.070583+0000","last_active":"2024-02-14T17:42:39.300965+0000","last_peered":"2024-02-14T17:42:39.300965+0000","last_clean":"2024-02-14T17:42:39.300965+0000","last_became_active":"2024-02-14T17:42:04.070263+0000","last_became_peered":"2024-02-14T17:42:04.070263+0000","last_unstale":"2024-02-14T17:42:39.300965+0000","last_undegraded":"2024-02-14T17:42:39.300965+0000","last_fullsized":"2024-02-14T17:42:39.300965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:52:55.132341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353334+0000","last_change":"2024-02-14T17:42:02.074131+0000","last_active":"2024-02-14T17:42:39.353334+0000","last_peered":"2024-02-14T17:42:39.353334+0000","last_clean":"2024-02-14T17:42:39.353334+0000","last_became_active":"2024-02-14T17:42:02.073717+0000","last_became_peered":"2024-02-14T17:42:02.073717+0000","last_unstale":"2024-02-14T17:42:39.353334+0000","last_undegraded":"2024-02-14T17:42:39.353334+0000","last_fullsized":"2024-02-14T17:42:39.353334+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:20:32.123349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353736+0000","last_change":"2024-02-14T17:41:58.044532+0000","last_active":"2024-02-14T17:42:39.353736+0000","last_peered":"2024-02-14T17:42:39.353736+0000","last_clean":"2024-02-14T17:42:39.353736+0000","last_became_active":"2024-02-14T17:41:58.044186+0000","last_became_peered":"2024-02-14T17:41:58.044186+0000","last_unstale":"2024-02-14T17:42:39.353736+0000","last_undegraded":"2024-02-14T17:42:39.353736+0000","last_fullsized":"2024-02-14T17:42:39.353736+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:48:22.993568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.301926+0000","last_change":"2024-02-14T17:42:00.044476+0000","last_active":"2024-02-14T17:42:39.301926+0000","last_peered":"2024-02-14T17:42:39.301926+0000","last_clean":"2024-02-14T17:42:39.301926+0000","last_became_active":"2024-02-14T17:42:00.044171+0000","last_became_peered":"2024-02-14T17:42:00.044171+0000","last_unstale":"2024-02-14T17:42:39.301926+0000","last_undegraded":"2024-02-14T17:42:39.301926+0000","last_fullsized":"2024-02-14T17:42:39.301926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:58:11.573231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354740+0000","last_change":"2024-02-14T17:42:04.079226+0000","last_active":"2024-02-14T17:42:39.354740+0000","last_peered":"2024-02-14T17:42:39.354740+0000","last_clean":"2024-02-14T17:42:39.354740+0000","last_became_active":"2024-02-14T17:42:04.078880+0000","last_became_peered":"2024-02-14T17:42:04.078880+0000","last_unstale":"2024-02-14T17:42:39.354740+0000","last_undegraded":"2024-02-14T17:42:39.354740+0000","last_fullsized":"2024-02-14T17:42:39.354740+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:37:52.069229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353415+0000","last_change":"2024-02-14T17:42:01.058940+0000","last_active":"2024-02-14T17:42:39.353415+0000","last_peered":"2024-02-14T17:42:39.353415+0000","last_clean":"2024-02-14T17:42:39.353415+0000","last_became_active":"2024-02-14T17:41:57.028333+0000","last_became_peered":"2024-02-14T17:41:57.028333+0000","last_unstale":"2024-02-14T17:42:39.353415+0000","last_undegraded":"2024-02-14T17:42:39.353415+0000","last_fullsized":"2024-02-14T17:42:39.353415+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_clean_scrub_stamp":"2024-02-14T17:41:56.017577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:53:06.948547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001143158,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353947+0000","last_change":"2024-02-14T17:41:58.037715+0000","last_active":"2024-02-14T17:42:39.353947+0000","last_peered":"2024-02-14T17:42:39.353947+0000","last_clean":"2024-02-14T17:42:39.353947+0000","last_became_active":"2024-02-14T17:41:58.037381+0000","last_became_peered":"2024-02-14T17:41:58.037381+0000","last_unstale":"2024-02-14T17:42:39.353947+0000","last_undegraded":"2024-02-14T17:42:39.353947+0000","last_fullsized":"2024-02-14T17:42:39.353947+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:59:36.165191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"70'28","reported_seq":93,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:08.208752+0000","last_change":"2024-02-14T17:42:00.053760+0000","last_active":"2024-02-14T17:43:08.208752+0000","last_peered":"2024-02-14T17:43:08.208752+0000","last_clean":"2024-02-14T17:43:08.208752+0000","last_became_active":"2024-02-14T17:42:00.053450+0000","last_became_peered":"2024-02-14T17:42:00.053450+0000","last_unstale":"2024-02-14T17:43:08.208752+0000","last_undegraded":"2024-02-14T17:43:08.208752+0000","last_fullsized":"2024-02-14T17:43:08.208752+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:11:55.086243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.301634+0000","last_change":"2024-02-14T17:42:02.067151+0000","last_active":"2024-02-14T17:42:39.301634+0000","last_peered":"2024-02-14T17:42:39.301634+0000","last_clean":"2024-02-14T17:42:39.301634+0000","last_became_active":"2024-02-14T17:42:02.066280+0000","last_became_peered":"2024-02-14T17:42:02.066280+0000","last_unstale":"2024-02-14T17:42:39.301634+0000","last_undegraded":"2024-02-14T17:42:39.301634+0000","last_fullsized":"2024-02-14T17:42:39.301634+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:04:27.169218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305215+0000","last_change":"2024-02-14T17:42:04.355594+0000","last_active":"2024-02-14T17:42:39.305215+0000","last_peered":"2024-02-14T17:42:39.305215+0000","last_clean":"2024-02-14T17:42:39.305215+0000","last_became_active":"2024-02-14T17:42:04.355262+0000","last_became_peered":"2024-02-14T17:42:04.355262+0000","last_unstale":"2024-02-14T17:42:39.305215+0000","last_undegraded":"2024-02-14T17:42:39.305215+0000","last_fullsized":"2024-02-14T17:42:39.305215+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:20:49.322891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354538+0000","last_change":"2024-02-14T17:41:58.158080+0000","last_active":"2024-02-14T17:42:39.354538+0000","last_peered":"2024-02-14T17:42:39.354538+0000","last_clean":"2024-02-14T17:42:39.354538+0000","last_became_active":"2024-02-14T17:41:58.157305+0000","last_became_peered":"2024-02-14T17:41:58.157305+0000","last_unstale":"2024-02-14T17:42:39.354538+0000","last_undegraded":"2024-02-14T17:42:39.354538+0000","last_fullsized":"2024-02-14T17:42:39.354538+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:20:00.476900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353051+0000","last_change":"2024-02-14T17:42:01.053171+0000","last_active":"2024-02-14T17:42:39.353051+0000","last_peered":"2024-02-14T17:42:39.353051+0000","last_clean":"2024-02-14T17:42:39.353051+0000","last_became_active":"2024-02-14T17:41:57.029052+0000","last_became_peered":"2024-02-14T17:41:57.029052+0000","last_unstale":"2024-02-14T17:42:39.353051+0000","last_undegraded":"2024-02-14T17:42:39.353051+0000","last_fullsized":"2024-02-14T17:42:39.353051+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_clean_scrub_stamp":"2024-02-14T17:41:56.017577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:53:20.653665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058337500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"70'13","reported_seq":56,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353855+0000","last_change":"2024-02-14T17:42:00.048089+0000","last_active":"2024-02-14T17:42:39.353855+0000","last_peered":"2024-02-14T17:42:39.353855+0000","last_clean":"2024-02-14T17:42:39.353855+0000","last_became_active":"2024-02-14T17:42:00.047771+0000","last_became_peered":"2024-02-14T17:42:00.047771+0000","last_unstale":"2024-02-14T17:42:39.353855+0000","last_undegraded":"2024-02-14T17:42:39.353855+0000","last_fullsized":"2024-02-14T17:42:39.353855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:38:06.471250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353999+0000","last_change":"2024-02-14T17:42:02.066483+0000","last_active":"2024-02-14T17:42:39.353999+0000","last_peered":"2024-02-14T17:42:39.353999+0000","last_clean":"2024-02-14T17:42:39.353999+0000","last_became_active":"2024-02-14T17:42:02.066155+0000","last_became_peered":"2024-02-14T17:42:02.066155+0000","last_unstale":"2024-02-14T17:42:39.353999+0000","last_undegraded":"2024-02-14T17:42:39.353999+0000","last_fullsized":"2024-02-14T17:42:39.353999+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:20:17.137160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354446+0000","last_change":"2024-02-14T17:42:04.072898+0000","last_active":"2024-02-14T17:42:39.354446+0000","last_peered":"2024-02-14T17:42:39.354446+0000","last_clean":"2024-02-14T17:42:39.354446+0000","last_became_active":"2024-02-14T17:42:04.072661+0000","last_became_peered":"2024-02-14T17:42:04.072661+0000","last_unstale":"2024-02-14T17:42:39.354446+0000","last_undegraded":"2024-02-14T17:42:39.354446+0000","last_fullsized":"2024-02-14T17:42:39.354446+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:48:56.117293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'128","reported_seq":279,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:41.398732+0000","last_change":"2024-02-14T17:41:48.169790+0000","last_active":"2024-02-14T17:42:41.398732+0000","last_peered":"2024-02-14T17:42:41.398732+0000","last_clean":"2024-02-14T17:42:41.398732+0000","last_became_active":"2024-02-14T17:41:48.169398+0000","last_became_peered":"2024-02-14T17:41:48.169398+0000","last_unstale":"2024-02-14T17:42:41.398732+0000","last_undegraded":"2024-02-14T17:42:41.398732+0000","last_fullsized":"2024-02-14T17:42:41.398732+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:39:39.206620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:39:39.206620+0000","last_clean_scrub_stamp":"2024-02-14T17:39:39.206620+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:09:05.151527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354255+0000","last_change":"2024-02-14T17:41:58.038148+0000","last_active":"2024-02-14T17:42:39.354255+0000","last_peered":"2024-02-14T17:42:39.354255+0000","last_clean":"2024-02-14T17:42:39.354255+0000","last_became_active":"2024-02-14T17:41:58.037856+0000","last_became_peered":"2024-02-14T17:41:58.037856+0000","last_unstale":"2024-02-14T17:42:39.354255+0000","last_undegraded":"2024-02-14T17:42:39.354255+0000","last_fullsized":"2024-02-14T17:42:39.354255+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:45:31.726044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"70'16","reported_seq":67,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:08.208951+0000","last_change":"2024-02-14T17:42:00.140373+0000","last_active":"2024-02-14T17:43:08.208951+0000","last_peered":"2024-02-14T17:43:08.208951+0000","last_clean":"2024-02-14T17:43:08.208951+0000","last_became_active":"2024-02-14T17:42:00.140055+0000","last_became_peered":"2024-02-14T17:42:00.140055+0000","last_unstale":"2024-02-14T17:43:08.208951+0000","last_undegraded":"2024-02-14T17:43:08.208951+0000","last_fullsized":"2024-02-14T17:43:08.208951+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:54:59.678414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305967+0000","last_change":"2024-02-14T17:42:02.070709+0000","last_active":"2024-02-14T17:42:39.305967+0000","last_peered":"2024-02-14T17:42:39.305967+0000","last_clean":"2024-02-14T17:42:39.305967+0000","last_became_active":"2024-02-14T17:42:02.070402+0000","last_became_peered":"2024-02-14T17:42:02.070402+0000","last_unstale":"2024-02-14T17:42:39.305967+0000","last_undegraded":"2024-02-14T17:42:39.305967+0000","last_fullsized":"2024-02-14T17:42:39.305967+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:22:09.231236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353555+0000","last_change":"2024-02-14T17:42:04.079131+0000","last_active":"2024-02-14T17:42:39.353555+0000","last_peered":"2024-02-14T17:42:39.353555+0000","last_clean":"2024-02-14T17:42:39.353555+0000","last_became_active":"2024-02-14T17:42:04.078748+0000","last_became_peered":"2024-02-14T17:42:04.078748+0000","last_unstale":"2024-02-14T17:42:39.353555+0000","last_undegraded":"2024-02-14T17:42:39.353555+0000","last_fullsized":"2024-02-14T17:42:39.353555+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:29:58.643121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.301158+0000","last_change":"2024-02-14T17:41:58.033841+0000","last_active":"2024-02-14T17:42:39.301158+0000","last_peered":"2024-02-14T17:42:39.301158+0000","last_clean":"2024-02-14T17:42:39.301158+0000","last_became_active":"2024-02-14T17:41:58.033567+0000","last_became_peered":"2024-02-14T17:41:58.033567+0000","last_unstale":"2024-02-14T17:42:39.301158+0000","last_undegraded":"2024-02-14T17:42:39.301158+0000","last_fullsized":"2024-02-14T17:42:39.301158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:17:07.841067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"71'5","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:13.905913+0000","last_change":"2024-02-14T17:42:01.059673+0000","last_active":"2024-02-14T17:43:13.905913+0000","last_peered":"2024-02-14T17:43:13.905913+0000","last_clean":"2024-02-14T17:43:13.905913+0000","last_became_active":"2024-02-14T17:41:57.028428+0000","last_became_peered":"2024-02-14T17:41:57.028428+0000","last_unstale":"2024-02-14T17:43:13.905913+0000","last_undegraded":"2024-02-14T17:43:13.905913+0000","last_fullsized":"2024-02-14T17:43:13.905913+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_clean_scrub_stamp":"2024-02-14T17:41:56.017577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:13:54.954585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022872890000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"70'12","reported_seq":47,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302347+0000","last_change":"2024-02-14T17:42:00.045240+0000","last_active":"2024-02-14T17:42:39.302347+0000","last_peered":"2024-02-14T17:42:39.302347+0000","last_clean":"2024-02-14T17:42:39.302347+0000","last_became_active":"2024-02-14T17:42:00.044171+0000","last_became_peered":"2024-02-14T17:42:00.044171+0000","last_unstale":"2024-02-14T17:42:39.302347+0000","last_undegraded":"2024-02-14T17:42:39.302347+0000","last_fullsized":"2024-02-14T17:42:39.302347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:04:55.458971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354680+0000","last_change":"2024-02-14T17:42:02.071968+0000","last_active":"2024-02-14T17:42:39.354680+0000","last_peered":"2024-02-14T17:42:39.354680+0000","last_clean":"2024-02-14T17:42:39.354680+0000","last_became_active":"2024-02-14T17:42:02.071641+0000","last_became_peered":"2024-02-14T17:42:02.071641+0000","last_unstale":"2024-02-14T17:42:39.354680+0000","last_undegraded":"2024-02-14T17:42:39.354680+0000","last_fullsized":"2024-02-14T17:42:39.354680+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:45:37.042480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354276+0000","last_change":"2024-02-14T17:42:04.355235+0000","last_active":"2024-02-14T17:42:39.354276+0000","last_peered":"2024-02-14T17:42:39.354276+0000","last_clean":"2024-02-14T17:42:39.354276+0000","last_became_active":"2024-02-14T17:42:04.354767+0000","last_became_peered":"2024-02-14T17:42:04.354767+0000","last_unstale":"2024-02-14T17:42:39.354276+0000","last_undegraded":"2024-02-14T17:42:39.354276+0000","last_fullsized":"2024-02-14T17:42:39.354276+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:39:56.384013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353684+0000","last_change":"2024-02-14T17:41:58.156156+0000","last_active":"2024-02-14T17:42:39.353684+0000","last_peered":"2024-02-14T17:42:39.353684+0000","last_clean":"2024-02-14T17:42:39.353684+0000","last_became_active":"2024-02-14T17:41:58.155754+0000","last_became_peered":"2024-02-14T17:41:58.155754+0000","last_unstale":"2024-02-14T17:42:39.353684+0000","last_undegraded":"2024-02-14T17:42:39.353684+0000","last_fullsized":"2024-02-14T17:42:39.353684+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:03:47.328120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"70'19","reported_seq":65,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302177+0000","last_change":"2024-02-14T17:42:00.140276+0000","last_active":"2024-02-14T17:42:39.302177+0000","last_peered":"2024-02-14T17:42:39.302177+0000","last_clean":"2024-02-14T17:42:39.302177+0000","last_became_active":"2024-02-14T17:42:00.139558+0000","last_became_peered":"2024-02-14T17:42:00.139558+0000","last_unstale":"2024-02-14T17:42:39.302177+0000","last_undegraded":"2024-02-14T17:42:39.302177+0000","last_fullsized":"2024-02-14T17:42:39.302177+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:09:21.737718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303463+0000","last_change":"2024-02-14T17:42:02.063057+0000","last_active":"2024-02-14T17:42:39.303463+0000","last_peered":"2024-02-14T17:42:39.303463+0000","last_clean":"2024-02-14T17:42:39.303463+0000","last_became_active":"2024-02-14T17:42:02.062781+0000","last_became_peered":"2024-02-14T17:42:02.062781+0000","last_unstale":"2024-02-14T17:42:39.303463+0000","last_undegraded":"2024-02-14T17:42:39.303463+0000","last_fullsized":"2024-02-14T17:42:39.303463+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:00:56.815449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.300965+0000","last_change":"2024-02-14T17:42:04.080204+0000","last_active":"2024-02-14T17:42:39.300965+0000","last_peered":"2024-02-14T17:42:39.300965+0000","last_clean":"2024-02-14T17:42:39.300965+0000","last_became_active":"2024-02-14T17:42:04.079811+0000","last_became_peered":"2024-02-14T17:42:04.079811+0000","last_unstale":"2024-02-14T17:42:39.300965+0000","last_undegraded":"2024-02-14T17:42:39.300965+0000","last_fullsized":"2024-02-14T17:42:39.300965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:37:45.857138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353964+0000","last_change":"2024-02-14T17:41:58.155985+0000","last_active":"2024-02-14T17:42:39.353964+0000","last_peered":"2024-02-14T17:42:39.353964+0000","last_clean":"2024-02-14T17:42:39.353964+0000","last_became_active":"2024-02-14T17:41:58.155597+0000","last_became_peered":"2024-02-14T17:41:58.155597+0000","last_unstale":"2024-02-14T17:42:39.353964+0000","last_undegraded":"2024-02-14T17:42:39.353964+0000","last_fullsized":"2024-02-14T17:42:39.353964+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:02.355959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354218+0000","last_change":"2024-02-14T17:42:00.050062+0000","last_active":"2024-02-14T17:42:39.354218+0000","last_peered":"2024-02-14T17:42:39.354218+0000","last_clean":"2024-02-14T17:42:39.354218+0000","last_became_active":"2024-02-14T17:42:00.049028+0000","last_became_peered":"2024-02-14T17:42:00.049028+0000","last_unstale":"2024-02-14T17:42:39.354218+0000","last_undegraded":"2024-02-14T17:42:39.354218+0000","last_fullsized":"2024-02-14T17:42:39.354218+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:29:37.283830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:08.208767+0000","last_change":"2024-02-14T17:42:02.060560+0000","last_active":"2024-02-14T17:43:08.208767+0000","last_peered":"2024-02-14T17:43:08.208767+0000","last_clean":"2024-02-14T17:43:08.208767+0000","last_became_active":"2024-02-14T17:42:02.060273+0000","last_became_peered":"2024-02-14T17:42:02.060273+0000","last_unstale":"2024-02-14T17:43:08.208767+0000","last_undegraded":"2024-02-14T17:43:08.208767+0000","last_fullsized":"2024-02-14T17:43:08.208767+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:18:11.905872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304537+0000","last_change":"2024-02-14T17:42:04.356528+0000","last_active":"2024-02-14T17:42:39.304537+0000","last_peered":"2024-02-14T17:42:39.304537+0000","last_clean":"2024-02-14T17:42:39.304537+0000","last_became_active":"2024-02-14T17:42:04.356098+0000","last_became_peered":"2024-02-14T17:42:04.356098+0000","last_unstale":"2024-02-14T17:42:39.304537+0000","last_undegraded":"2024-02-14T17:42:39.304537+0000","last_fullsized":"2024-02-14T17:42:39.304537+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:17:24.241704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302489+0000","last_change":"2024-02-14T17:41:58.032968+0000","last_active":"2024-02-14T17:42:39.302489+0000","last_peered":"2024-02-14T17:42:39.302489+0000","last_clean":"2024-02-14T17:42:39.302489+0000","last_became_active":"2024-02-14T17:41:58.032684+0000","last_became_peered":"2024-02-14T17:41:58.032684+0000","last_unstale":"2024-02-14T17:42:39.302489+0000","last_undegraded":"2024-02-14T17:42:39.302489+0000","last_fullsized":"2024-02-14T17:42:39.302489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:47:59.156383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"70'14","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353643+0000","last_change":"2024-02-14T17:42:00.046358+0000","last_active":"2024-02-14T17:42:39.353643+0000","last_peered":"2024-02-14T17:42:39.353643+0000","last_clean":"2024-02-14T17:42:39.353643+0000","last_became_active":"2024-02-14T17:42:00.045634+0000","last_became_peered":"2024-02-14T17:42:00.045634+0000","last_unstale":"2024-02-14T17:42:39.353643+0000","last_undegraded":"2024-02-14T17:42:39.353643+0000","last_fullsized":"2024-02-14T17:42:39.353643+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:35:33.676299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:08.208764+0000","last_change":"2024-02-14T17:42:02.074810+0000","last_active":"2024-02-14T17:43:08.208764+0000","last_peered":"2024-02-14T17:43:08.208764+0000","last_clean":"2024-02-14T17:43:08.208764+0000","last_became_active":"2024-02-14T17:42:02.073089+0000","last_became_peered":"2024-02-14T17:42:02.073089+0000","last_unstale":"2024-02-14T17:43:08.208764+0000","last_undegraded":"2024-02-14T17:43:08.208764+0000","last_fullsized":"2024-02-14T17:43:08.208764+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:04:20.865557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353365+0000","last_change":"2024-02-14T17:42:04.079675+0000","last_active":"2024-02-14T17:42:39.353365+0000","last_peered":"2024-02-14T17:42:39.353365+0000","last_clean":"2024-02-14T17:42:39.353365+0000","last_became_active":"2024-02-14T17:42:04.079387+0000","last_became_peered":"2024-02-14T17:42:04.079387+0000","last_unstale":"2024-02-14T17:42:39.353365+0000","last_undegraded":"2024-02-14T17:42:39.353365+0000","last_fullsized":"2024-02-14T17:42:39.353365+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:01:31.759894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354841+0000","last_change":"2024-02-14T17:41:58.031823+0000","last_active":"2024-02-14T17:42:39.354841+0000","last_peered":"2024-02-14T17:42:39.354841+0000","last_clean":"2024-02-14T17:42:39.354841+0000","last_became_active":"2024-02-14T17:41:58.031463+0000","last_became_peered":"2024-02-14T17:41:58.031463+0000","last_unstale":"2024-02-14T17:42:39.354841+0000","last_undegraded":"2024-02-14T17:42:39.354841+0000","last_fullsized":"2024-02-14T17:42:39.354841+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:08:21.970694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"70'18","reported_seq":61,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353660+0000","last_change":"2024-02-14T17:42:00.141209+0000","last_active":"2024-02-14T17:42:39.353660+0000","last_peered":"2024-02-14T17:42:39.353660+0000","last_clean":"2024-02-14T17:42:39.353660+0000","last_became_active":"2024-02-14T17:42:00.140912+0000","last_became_peered":"2024-02-14T17:42:00.140912+0000","last_unstale":"2024-02-14T17:42:39.353660+0000","last_undegraded":"2024-02-14T17:42:39.353660+0000","last_fullsized":"2024-02-14T17:42:39.353660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:58:51.413490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353284+0000","last_change":"2024-02-14T17:42:02.066962+0000","last_active":"2024-02-14T17:42:39.353284+0000","last_peered":"2024-02-14T17:42:39.353284+0000","last_clean":"2024-02-14T17:42:39.353284+0000","last_became_active":"2024-02-14T17:42:02.065889+0000","last_became_peered":"2024-02-14T17:42:02.065889+0000","last_unstale":"2024-02-14T17:42:39.353284+0000","last_undegraded":"2024-02-14T17:42:39.353284+0000","last_fullsized":"2024-02-14T17:42:39.353284+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:09:12.526329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353790+0000","last_change":"2024-02-14T17:42:04.072382+0000","last_active":"2024-02-14T17:42:39.353790+0000","last_peered":"2024-02-14T17:42:39.353790+0000","last_clean":"2024-02-14T17:42:39.353790+0000","last_became_active":"2024-02-14T17:42:04.072126+0000","last_became_peered":"2024-02-14T17:42:04.072126+0000","last_unstale":"2024-02-14T17:42:39.353790+0000","last_undegraded":"2024-02-14T17:42:39.353790+0000","last_fullsized":"2024-02-14T17:42:39.353790+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:33:07.266337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:08.208978+0000","last_change":"2024-02-14T17:42:02.060502+0000","last_active":"2024-02-14T17:43:08.208978+0000","last_peered":"2024-02-14T17:43:08.208978+0000","last_clean":"2024-02-14T17:43:08.208978+0000","last_became_active":"2024-02-14T17:42:02.060196+0000","last_became_peered":"2024-02-14T17:42:02.060196+0000","last_unstale":"2024-02-14T17:43:08.208978+0000","last_undegraded":"2024-02-14T17:43:08.208978+0000","last_fullsized":"2024-02-14T17:43:08.208978+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:00:27.871131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353570+0000","last_change":"2024-02-14T17:41:58.036259+0000","last_active":"2024-02-14T17:42:39.353570+0000","last_peered":"2024-02-14T17:42:39.353570+0000","last_clean":"2024-02-14T17:42:39.353570+0000","last_became_active":"2024-02-14T17:41:58.035918+0000","last_became_peered":"2024-02-14T17:41:58.035918+0000","last_unstale":"2024-02-14T17:42:39.353570+0000","last_undegraded":"2024-02-14T17:42:39.353570+0000","last_fullsized":"2024-02-14T17:42:39.353570+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:12:06.124705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353945+0000","last_change":"2024-02-14T17:42:00.053171+0000","last_active":"2024-02-14T17:42:39.353945+0000","last_peered":"2024-02-14T17:42:39.353945+0000","last_clean":"2024-02-14T17:42:39.353945+0000","last_became_active":"2024-02-14T17:42:00.052841+0000","last_became_peered":"2024-02-14T17:42:00.052841+0000","last_unstale":"2024-02-14T17:42:39.353945+0000","last_undegraded":"2024-02-14T17:42:39.353945+0000","last_fullsized":"2024-02-14T17:42:39.353945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:54:39.633547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354582+0000","last_change":"2024-02-14T17:42:04.355576+0000","last_active":"2024-02-14T17:42:39.354582+0000","last_peered":"2024-02-14T17:42:39.354582+0000","last_clean":"2024-02-14T17:42:39.354582+0000","last_became_active":"2024-02-14T17:42:04.355227+0000","last_became_peered":"2024-02-14T17:42:04.355227+0000","last_unstale":"2024-02-14T17:42:39.354582+0000","last_undegraded":"2024-02-14T17:42:39.354582+0000","last_fullsized":"2024-02-14T17:42:39.354582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:29:37.481330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354671+0000","last_change":"2024-02-14T17:42:02.063035+0000","last_active":"2024-02-14T17:42:39.354671+0000","last_peered":"2024-02-14T17:42:39.354671+0000","last_clean":"2024-02-14T17:42:39.354671+0000","last_became_active":"2024-02-14T17:42:02.062653+0000","last_became_peered":"2024-02-14T17:42:02.062653+0000","last_unstale":"2024-02-14T17:42:39.354671+0000","last_undegraded":"2024-02-14T17:42:39.354671+0000","last_fullsized":"2024-02-14T17:42:39.354671+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:00:23.049275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353386+0000","last_change":"2024-02-14T17:41:58.156296+0000","last_active":"2024-02-14T17:42:39.353386+0000","last_peered":"2024-02-14T17:42:39.353386+0000","last_clean":"2024-02-14T17:42:39.353386+0000","last_became_active":"2024-02-14T17:41:58.155924+0000","last_became_peered":"2024-02-14T17:41:58.155924+0000","last_unstale":"2024-02-14T17:42:39.353386+0000","last_undegraded":"2024-02-14T17:42:39.353386+0000","last_fullsized":"2024-02-14T17:42:39.353386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:18:47.062335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354454+0000","last_change":"2024-02-14T17:42:00.043521+0000","last_active":"2024-02-14T17:42:39.354454+0000","last_peered":"2024-02-14T17:42:39.354454+0000","last_clean":"2024-02-14T17:42:39.354454+0000","last_became_active":"2024-02-14T17:42:00.043124+0000","last_became_peered":"2024-02-14T17:42:00.043124+0000","last_unstale":"2024-02-14T17:42:39.354454+0000","last_undegraded":"2024-02-14T17:42:39.354454+0000","last_fullsized":"2024-02-14T17:42:39.354454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:36:49.304574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353613+0000","last_change":"2024-02-14T17:42:04.080634+0000","last_active":"2024-02-14T17:42:39.353613+0000","last_peered":"2024-02-14T17:42:39.353613+0000","last_clean":"2024-02-14T17:42:39.353613+0000","last_became_active":"2024-02-14T17:42:04.080127+0000","last_became_peered":"2024-02-14T17:42:04.080127+0000","last_unstale":"2024-02-14T17:42:39.353613+0000","last_undegraded":"2024-02-14T17:42:39.353613+0000","last_fullsized":"2024-02-14T17:42:39.353613+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:22:15.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354085+0000","last_change":"2024-02-14T17:42:02.072161+0000","last_active":"2024-02-14T17:42:39.354085+0000","last_peered":"2024-02-14T17:42:39.354085+0000","last_clean":"2024-02-14T17:42:39.354085+0000","last_became_active":"2024-02-14T17:42:02.071813+0000","last_became_peered":"2024-02-14T17:42:02.071813+0000","last_unstale":"2024-02-14T17:42:39.354085+0000","last_undegraded":"2024-02-14T17:42:39.354085+0000","last_fullsized":"2024-02-14T17:42:39.354085+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:27:07.653643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303004+0000","last_change":"2024-02-14T17:41:58.032975+0000","last_active":"2024-02-14T17:42:39.303004+0000","last_peered":"2024-02-14T17:42:39.303004+0000","last_clean":"2024-02-14T17:42:39.303004+0000","last_became_active":"2024-02-14T17:41:58.032623+0000","last_became_peered":"2024-02-14T17:41:58.032623+0000","last_unstale":"2024-02-14T17:42:39.303004+0000","last_undegraded":"2024-02-14T17:42:39.303004+0000","last_fullsized":"2024-02-14T17:42:39.303004+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:36:56.678002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"70'17","reported_seq":57,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354365+0000","last_change":"2024-02-14T17:42:00.046141+0000","last_active":"2024-02-14T17:42:39.354365+0000","last_peered":"2024-02-14T17:42:39.354365+0000","last_clean":"2024-02-14T17:42:39.354365+0000","last_became_active":"2024-02-14T17:42:00.045716+0000","last_became_peered":"2024-02-14T17:42:00.045716+0000","last_unstale":"2024-02-14T17:42:39.354365+0000","last_undegraded":"2024-02-14T17:42:39.354365+0000","last_fullsized":"2024-02-14T17:42:39.354365+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:56:05.037366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354772+0000","last_change":"2024-02-14T17:42:04.074080+0000","last_active":"2024-02-14T17:42:39.354772+0000","last_peered":"2024-02-14T17:42:39.354772+0000","last_clean":"2024-02-14T17:42:39.354772+0000","last_became_active":"2024-02-14T17:42:04.073799+0000","last_became_peered":"2024-02-14T17:42:04.073799+0000","last_unstale":"2024-02-14T17:42:39.354772+0000","last_undegraded":"2024-02-14T17:42:39.354772+0000","last_fullsized":"2024-02-14T17:42:39.354772+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:50:38.076407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:08.208846+0000","last_change":"2024-02-14T17:42:02.065914+0000","last_active":"2024-02-14T17:43:08.208846+0000","last_peered":"2024-02-14T17:43:08.208846+0000","last_clean":"2024-02-14T17:43:08.208846+0000","last_became_active":"2024-02-14T17:42:02.065569+0000","last_became_peered":"2024-02-14T17:42:02.065569+0000","last_unstale":"2024-02-14T17:43:08.208846+0000","last_undegraded":"2024-02-14T17:43:08.208846+0000","last_fullsized":"2024-02-14T17:43:08.208846+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:16:00.343907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355251+0000","last_change":"2024-02-14T17:41:58.033631+0000","last_active":"2024-02-14T17:42:39.355251+0000","last_peered":"2024-02-14T17:42:39.355251+0000","last_clean":"2024-02-14T17:42:39.355251+0000","last_became_active":"2024-02-14T17:41:58.033337+0000","last_became_peered":"2024-02-14T17:41:58.033337+0000","last_unstale":"2024-02-14T17:42:39.355251+0000","last_undegraded":"2024-02-14T17:42:39.355251+0000","last_fullsized":"2024-02-14T17:42:39.355251+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:31:09.656319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354548+0000","last_change":"2024-02-14T17:42:00.050280+0000","last_active":"2024-02-14T17:42:39.354548+0000","last_peered":"2024-02-14T17:42:39.354548+0000","last_clean":"2024-02-14T17:42:39.354548+0000","last_became_active":"2024-02-14T17:42:00.049856+0000","last_became_peered":"2024-02-14T17:42:00.049856+0000","last_unstale":"2024-02-14T17:42:39.354548+0000","last_undegraded":"2024-02-14T17:42:39.354548+0000","last_fullsized":"2024-02-14T17:42:39.354548+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:42:39.563143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168597,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1331200,"data_stored":879723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53700000000000003}]},{"osd":1,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.42799999999999999}]},{"osd":2,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58399999999999996}]},{"osd":3,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55300000000000005}]},{"osd":4,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36199999999999999}]},{"osd":5,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.624}]},{"osd":6,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56999999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462936,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28300,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716852,"statfs":{"total":95995035648,"available":95966056448,"internally_reserved":0,"allocated":1314816,"data_stored":878685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Wed Feb 14 17:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Feb 14 17:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Feb 14 17:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47499999999999998}]},{"osd":4,"last update":"Wed Feb 14 17:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.37}]},{"osd":5,"last update":"Wed Feb 14 17:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.48899999999999999}]},{"osd":7,"last update":"Wed Feb 14 17:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.432}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31928,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713224,"statfs":{"total":95995035648,"available":95962341376,"internally_reserved":0,"allocated":765952,"data_stored":289173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:42:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33700000000000002,"15min":0.315},"min":{"1min":0.24299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.34599999999999997,"15min":0.315},"min":{"1min":0.29699999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.28699999999999998}]},{"osd":2,"last update":"Wed Feb 14 17:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.39000000000000001,"15min":0.37},"min":{"1min":0.248,"5min":0.221,"15min":0.221},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.441,"15min":0.41499999999999998},"min":{"1min":0.34699999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.36399999999999999}]},{"osd":3,"last update":"Wed Feb 14 17:43:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.439,"15min":0.435},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.434,"15min":0.436},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.27300000000000002}]},{"osd":4,"last update":"Wed Feb 14 17:42:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.47799999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.65900000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.46100000000000002}]},{"osd":5,"last update":"Wed Feb 14 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.0469999999999997,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.47399999999999998}]},{"osd":6,"last update":"Wed Feb 14 17:42:22 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.497}]},{"osd":7,"last update":"Wed Feb 14 17:42:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.0110000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.42599999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1331200,"data_stored":878592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 17:42:15 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.51700000000000002,"15min":0.57599999999999996},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.54800000000000004,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.378,"15min":0.35199999999999998},"min":{"1min":0.25,"5min":0.187,"15min":0.187},"max":{"1min":0.60299999999999998,"5min":0.626,"15min":0.626},"last":0.30399999999999999}]},{"osd":2,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41899999999999998,"15min":0.40899999999999997},"min":{"1min":0.28599999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54500000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.253},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39000000000000001,"15min":0.38300000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed Feb 14 17:43:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.47699999999999998,"15min":0.49199999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65100000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.52300000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36799999999999999}]},{"osd":4,"last update":"Wed Feb 14 17:42:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.621,"15min":0.629},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80400000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.60099999999999998,"15min":0.60799999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82199999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.59299999999999997,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.67800000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.628,"15min":0.64000000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71199999999999997,"5min":0.876,"15min":0.876},"last":0.749}]},{"osd":6,"last update":"Wed Feb 14 17:42:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.70499999999999996}]},{"osd":7,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.76100000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31884,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713268,"statfs":{"total":95995035648,"available":95962386432,"internally_reserved":0,"allocated":720896,"data_stored":287874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40100000000000002,"15min":0.39800000000000002},"min":{"1min":0.27700000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.29499999999999998}]},{"osd":1,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41799999999999998,"15min":0.41599999999999998},"min":{"1min":0.26300000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.25600000000000001}]},{"osd":3,"last update":"Wed Feb 14 17:43:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44500000000000001,"15min":0.45300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.32100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67800000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Feb 14 17:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Feb 14 17:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.80200000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70799999999999996}]},{"osd":6,"last update":"Wed Feb 14 17:42:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.751}]},{"osd":7,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.72699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149735,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31920,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713232,"statfs":{"total":95995035648,"available":95962349568,"internally_reserved":0,"allocated":757760,"data_stored":288732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:43:06 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.378,"15min":0.378},"min":{"1min":0.23899999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.499,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.437,"15min":0.439},"min":{"1min":0.26500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.63200000000000001,"5min":0.746,"15min":0.746},"last":0.46300000000000002}]},{"osd":1,"last update":"Wed Feb 14 17:43:06 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.41899999999999998,"15min":0.42399999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.59399999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38300000000000001}]},{"osd":2,"last update":"Wed Feb 14 17:43:06 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46400000000000002,"15min":0.46200000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66500000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.504},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.46200000000000002,"15min":0.46800000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Wed Feb 14 17:42:31 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77700000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.84599999999999997}]},{"osd":5,"last update":"Wed Feb 14 17:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78600000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77600000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.88400000000000001}]},{"osd":6,"last update":"Wed Feb 14 17:42:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73499999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31908,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713244,"statfs":{"total":95995035648,"available":95962361856,"internally_reserved":0,"allocated":745472,"data_stored":288433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.622},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.40200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed Feb 14 17:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51700000000000002,"15min":0.51600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70899999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed Feb 14 17:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65300000000000002}]},{"osd":3,"last update":"Wed Feb 14 17:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.42199999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed Feb 14 17:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.29199999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.67200000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.39000000000000001,"15min":0.379},"min":{"1min":0.28699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.443}]},{"osd":6,"last update":"Wed Feb 14 17:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.39300000000000002}]},{"osd":7,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.50800000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724574,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27732,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717420,"statfs":{"total":95995035648,"available":95966638080,"internally_reserved":0,"allocated":729088,"data_stored":288752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.38300000000000001}]},{"osd":1,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76500000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.69499999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42199999999999999}]},{"osd":2,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.436},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.42299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.67300000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.40999999999999998}]},{"osd":3,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39000000000000001,"15min":0.38200000000000001},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.39200000000000002,"15min":0.378},"min":{"1min":0.33700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.35199999999999998}]},{"osd":6,"last update":"Wed Feb 14 17:42:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.318},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46800000000000003}]},{"osd":7,"last update":"Wed Feb 14 17:42:46 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T17:43:24.931 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-14T17:43:25.198 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:24 smithi178 ceph-mon[39811]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:25.199 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:24 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:25.199 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:24 smithi178 ceph-mon[30321]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:25.199 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:24 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:25.199 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 podman[78017]: 2024-02-14 17:43:25.12261379 +0000 UTC m=+0.101723771 volume create 9b0d8921dfb3160ac6fa266a23fc1c4876bec99703bfdebbc673f64459e714cd 2024-02-14T17:43:25.199 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 podman[78017]: 2024-02-14 17:43:25.039278477 +0000 UTC m=+0.018388460 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-02-14T17:43:25.199 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 podman[78017]: 2024-02-14 17:43:25.198096258 +0000 UTC m=+0.177206239 container create 8c8a8c324cce86cf34c6a36cef46b7d3b40bd9bc78f5b69b3ac4ed5b800967d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:43:25.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:24 smithi184 ceph-mon[33536]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:25.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:24 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:25.657 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:43:25.770 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:43:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:43:25.770 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 podman[78017]: 2024-02-14 17:43:25.509368772 +0000 UTC m=+0.488478753 container init 8c8a8c324cce86cf34c6a36cef46b7d3b40bd9bc78f5b69b3ac4ed5b800967d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:43:25.770 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 podman[78017]: 2024-02-14 17:43:25.51456131 +0000 UTC m=+0.493671292 container start 8c8a8c324cce86cf34c6a36cef46b7d3b40bd9bc78f5b69b3ac4ed5b800967d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:43:25.770 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78060]: ts=2024-02-14T17:43:25.526Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-14T17:43:25.770 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78060]: ts=2024-02-14T17:43:25.526Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-14T17:43:25.770 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78060]: ts=2024-02-14T17:43:25.527Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.178 port=9094 2024-02-14T17:43:25.770 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78060]: ts=2024-02-14T17:43:25.528Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-14T17:43:25.770 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78060]: ts=2024-02-14T17:43:25.551Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-14T17:43:25.771 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78060]: ts=2024-02-14T17:43:25.552Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-14T17:43:25.771 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78060]: ts=2024-02-14T17:43:25.554Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-14T17:43:25.771 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78060]: ts=2024-02-14T17:43:25.554Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-14T17:43:25.771 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 bash[78017]: 8c8a8c324cce86cf34c6a36cef46b7d3b40bd9bc78f5b69b3ac4ed5b800967d8 2024-02-14T17:43:25.771 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:25 smithi178 systemd[1]: Started Ceph alertmanager.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:43:26.020 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:43:25] ENGINE Bus STOPPING 2024-02-14T17:43:26.021 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:43:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T17:43:26.021 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:43:25] ENGINE Bus STOPPED 2024-02-14T17:43:26.021 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:43:25] ENGINE Bus STARTING 2024-02-14T17:43:26.021 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:43:25] ENGINE Serving on http://:::9283 2024-02-14T17:43:26.021 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:43:25] ENGINE Bus STARTED 2024-02-14T17:43:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-14T17:43:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:43:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:43:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:26 smithi184 ceph-mon[33536]: Deploying daemon grafana.a on smithi184 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[39811]: Deploying daemon grafana.a on smithi184 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:26.977 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:26 smithi178 ceph-mon[30321]: Deploying daemon grafana.a on smithi184 2024-02-14T17:43:27.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.438+0000 7f3b971bc700 1 -- 172.21.15.178:0/2254984181 >> v1:172.21.15.178:6790/0 conn(0x7f3b901097e0 legacy=0x7f3b9010bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:27.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.438+0000 7f3b971bc700 1 -- 172.21.15.178:0/2254984181 shutdown_connections 2024-02-14T17:43:27.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.438+0000 7f3b971bc700 1 -- 172.21.15.178:0/2254984181 >> 172.21.15.178:0/2254984181 conn(0x7f3b900fc440 msgr2=0x7f3b900fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:27.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.438+0000 7f3b971bc700 1 -- 172.21.15.178:0/2254984181 shutdown_connections 2024-02-14T17:43:27.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.438+0000 7f3b971bc700 1 -- 172.21.15.178:0/2254984181 wait complete. 2024-02-14T17:43:27.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.439+0000 7f3b971bc700 1 Processor -- start 2024-02-14T17:43:27.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.439+0000 7f3b971bc700 1 -- start start 2024-02-14T17:43:27.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.439+0000 7f3b971bc700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3b901a6910 con 0x7f3b90106930 2024-02-14T17:43:27.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.439+0000 7f3b971bc700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3b901a7ab0 con 0x7f3b901097e0 2024-02-14T17:43:27.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.439+0000 7f3b971bc700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3b901a8cb0 con 0x7f3b9010d3b0 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.439+0000 7f3b94f58700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f3b90106930 0x7f3b9010ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:46368/0 (socket says 172.21.15.178:46368) 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.439+0000 7f3b94f58700 1 -- 172.21.15.178:0/778231243 learned_addr learned my addr 172.21.15.178:0/778231243 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094436285 0 0) 0x7f3b901a8cb0 con 0x7f3b9010d3b0 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b800035b0 con 0x7f3b9010d3b0 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307055337 0 0) 0x7f3b901a6910 con 0x7f3b90106930 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b901a8cb0 con 0x7f3b90106930 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824457984 0 0) 0x7f3b901a7ab0 con 0x7f3b901097e0 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b901a6910 con 0x7f3b901097e0 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 207316169 0 0) 0x7f3b800035b0 con 0x7f3b9010d3b0 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3b901a7ab0 con 0x7f3b9010d3b0 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f3b88001bb0 con 0x7f3b9010d3b0 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3120673153 0 0) 0x7f3b901a8cb0 con 0x7f3b90106930 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3b800035b0 con 0x7f3b90106930 2024-02-14T17:43:27.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1453537270 0 0) 0x7f3b901a6910 con 0x7f3b901097e0 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3b901a8cb0 con 0x7f3b901097e0 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f3b84001870 con 0x7f3b90106930 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f3b7c0043a0 con 0x7f3b901097e0 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4002109886 0 0) 0x7f3b901a7ab0 con 0x7f3b9010d3b0 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 >> v1:172.21.15.184:6789/0 conn(0x7f3b901097e0 legacy=0x7f3b901a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 >> v1:172.21.15.178:6789/0 conn(0x7f3b90106930 legacy=0x7f3b9010ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b901a9eb0 con 0x7f3b9010d3b0 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b971bc700 1 -- 172.21.15.178:0/778231243 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b901a6b00 con 0x7f3b9010d3b0 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f3b88003e90 con 0x7f3b9010d3b0 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f3b88004d50 con 0x7f3b9010d3b0 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.440+0000 7f3b971bc700 1 -- 172.21.15.178:0/778231243 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b901a6f80 con 0x7f3b9010d3b0 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.441+0000 7f3b971bc700 1 -- 172.21.15.178:0/778231243 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b90102070 con 0x7f3b9010d3b0 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.442+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f3b88003a00 con 0x7f3b9010d3b0 2024-02-14T17:43:27.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.442+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7f3b88087b90 con 0x7f3b9010d3b0 2024-02-14T17:43:27.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.447+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f3b88054e90 con 0x7f3b9010d3b0 2024-02-14T17:43:27.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.635+0000 7f3b971bc700 1 -- 172.21.15.178:0/778231243 --> v1:172.21.15.178:6800/4205070280 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b900685d0 con 0x7f3b8006ecd0 2024-02-14T17:43:27.637 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:27 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78060]: ts=2024-02-14T17:43:27.529Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000202544s 2024-02-14T17:43:27.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.645+0000 7f3b8dffb700 1 -- 172.21.15.178:0/778231243 <== mgr.14595 v1:172.21.15.178:6800/4205070280 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+383775 (unknown 2965378022 0 3291316301) 0x7f3b900685d0 con 0x7f3b8006ecd0 2024-02-14T17:43:27.651 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:43:27.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.649+0000 7f3b971bc700 1 -- 172.21.15.178:0/778231243 >> v1:172.21.15.178:6800/4205070280 conn(0x7f3b8006ecd0 legacy=0x7f3b80071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:27.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.649+0000 7f3b971bc700 1 -- 172.21.15.178:0/778231243 >> v1:172.21.15.178:6790/0 conn(0x7f3b9010d3b0 legacy=0x7f3b901a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:27.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.649+0000 7f3b971bc700 1 -- 172.21.15.178:0/778231243 shutdown_connections 2024-02-14T17:43:27.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.649+0000 7f3b971bc700 1 -- 172.21.15.178:0/778231243 >> 172.21.15.178:0/778231243 conn(0x7f3b900fc440 msgr2=0x7f3b90110040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:27.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.649+0000 7f3b971bc700 1 -- 172.21.15.178:0/778231243 shutdown_connections 2024-02-14T17:43:27.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:27.649+0000 7f3b971bc700 1 -- 172.21.15.178:0/778231243 wait complete. 2024-02-14T17:43:27.654 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-02-14T17:43:28.303 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-02-14T17:43:27.320749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1014,"num_read_kb":860,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":1771544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":248624,"kb_used_data":7516,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":749712592,"statfs":{"total":767960285184,"available":767705694208,"internally_reserved":0,"allocated":7696384,"data_stored":4079964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":246795830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002686"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353263+0000","last_change":"2024-02-14T17:42:04.072035+0000","last_active":"2024-02-14T17:42:39.353263+0000","last_peered":"2024-02-14T17:42:39.353263+0000","last_clean":"2024-02-14T17:42:39.353263+0000","last_became_active":"2024-02-14T17:42:04.071721+0000","last_became_peered":"2024-02-14T17:42:04.071721+0000","last_unstale":"2024-02-14T17:42:39.353263+0000","last_undegraded":"2024-02-14T17:42:39.353263+0000","last_fullsized":"2024-02-14T17:42:39.353263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:42:02.683826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354362+0000","last_change":"2024-02-14T17:42:02.068619+0000","last_active":"2024-02-14T17:42:39.354362+0000","last_peered":"2024-02-14T17:42:39.354362+0000","last_clean":"2024-02-14T17:42:39.354362+0000","last_became_active":"2024-02-14T17:42:02.068319+0000","last_became_peered":"2024-02-14T17:42:02.068319+0000","last_unstale":"2024-02-14T17:42:39.354362+0000","last_undegraded":"2024-02-14T17:42:39.354362+0000","last_fullsized":"2024-02-14T17:42:39.354362+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:15:44.584475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302130+0000","last_change":"2024-02-14T17:41:58.156367+0000","last_active":"2024-02-14T17:42:39.302130+0000","last_peered":"2024-02-14T17:42:39.302130+0000","last_clean":"2024-02-14T17:42:39.302130+0000","last_became_active":"2024-02-14T17:41:58.155580+0000","last_became_peered":"2024-02-14T17:41:58.155580+0000","last_unstale":"2024-02-14T17:42:39.302130+0000","last_undegraded":"2024-02-14T17:42:39.302130+0000","last_fullsized":"2024-02-14T17:42:39.302130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:23:00.873224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"70'9","reported_seq":61,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353248+0000","last_change":"2024-02-14T17:42:22.157602+0000","last_active":"2024-02-14T17:42:39.353248+0000","last_peered":"2024-02-14T17:42:39.353248+0000","last_clean":"2024-02-14T17:42:39.353248+0000","last_became_active":"2024-02-14T17:42:22.150824+0000","last_became_peered":"2024-02-14T17:42:22.150824+0000","last_unstale":"2024-02-14T17:42:39.353248+0000","last_undegraded":"2024-02-14T17:42:39.353248+0000","last_fullsized":"2024-02-14T17:42:39.353248+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:42:15.412332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355196+0000","last_change":"2024-02-14T17:42:04.356662+0000","last_active":"2024-02-14T17:42:39.355196+0000","last_peered":"2024-02-14T17:42:39.355196+0000","last_clean":"2024-02-14T17:42:39.355196+0000","last_became_active":"2024-02-14T17:42:04.355722+0000","last_became_peered":"2024-02-14T17:42:04.355722+0000","last_unstale":"2024-02-14T17:42:39.355196+0000","last_undegraded":"2024-02-14T17:42:39.355196+0000","last_fullsized":"2024-02-14T17:42:39.355196+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:48:51.796142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353383+0000","last_change":"2024-02-14T17:42:02.072928+0000","last_active":"2024-02-14T17:42:39.353383+0000","last_peered":"2024-02-14T17:42:39.353383+0000","last_clean":"2024-02-14T17:42:39.353383+0000","last_became_active":"2024-02-14T17:42:02.072647+0000","last_became_peered":"2024-02-14T17:42:02.072647+0000","last_unstale":"2024-02-14T17:42:39.353383+0000","last_undegraded":"2024-02-14T17:42:39.353383+0000","last_fullsized":"2024-02-14T17:42:39.353383+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:25:03.883605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355063+0000","last_change":"2024-02-14T17:41:58.155858+0000","last_active":"2024-02-14T17:42:39.355063+0000","last_peered":"2024-02-14T17:42:39.355063+0000","last_clean":"2024-02-14T17:42:39.355063+0000","last_became_active":"2024-02-14T17:41:58.155562+0000","last_became_peered":"2024-02-14T17:41:58.155562+0000","last_unstale":"2024-02-14T17:42:39.355063+0000","last_undegraded":"2024-02-14T17:42:39.355063+0000","last_fullsized":"2024-02-14T17:42:39.355063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:31:59.146624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354423+0000","last_change":"2024-02-14T17:42:00.052945+0000","last_active":"2024-02-14T17:42:39.354423+0000","last_peered":"2024-02-14T17:42:39.354423+0000","last_clean":"2024-02-14T17:42:39.354423+0000","last_became_active":"2024-02-14T17:42:00.052565+0000","last_became_peered":"2024-02-14T17:42:00.052565+0000","last_unstale":"2024-02-14T17:42:39.354423+0000","last_undegraded":"2024-02-14T17:42:39.354423+0000","last_fullsized":"2024-02-14T17:42:39.354423+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:55:28.405525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.301610+0000","last_change":"2024-02-14T17:42:04.070998+0000","last_active":"2024-02-14T17:42:39.301610+0000","last_peered":"2024-02-14T17:42:39.301610+0000","last_clean":"2024-02-14T17:42:39.301610+0000","last_became_active":"2024-02-14T17:42:04.070678+0000","last_became_peered":"2024-02-14T17:42:04.070678+0000","last_unstale":"2024-02-14T17:42:39.301610+0000","last_undegraded":"2024-02-14T17:42:39.301610+0000","last_fullsized":"2024-02-14T17:42:39.301610+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:12:56.679400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353411+0000","last_change":"2024-02-14T17:42:02.067757+0000","last_active":"2024-02-14T17:42:39.353411+0000","last_peered":"2024-02-14T17:42:39.353411+0000","last_clean":"2024-02-14T17:42:39.353411+0000","last_became_active":"2024-02-14T17:42:02.067110+0000","last_became_peered":"2024-02-14T17:42:02.067110+0000","last_unstale":"2024-02-14T17:42:39.353411+0000","last_undegraded":"2024-02-14T17:42:39.353411+0000","last_fullsized":"2024-02-14T17:42:39.353411+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:58:19.455017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353857+0000","last_change":"2024-02-14T17:41:58.042092+0000","last_active":"2024-02-14T17:42:39.353857+0000","last_peered":"2024-02-14T17:42:39.353857+0000","last_clean":"2024-02-14T17:42:39.353857+0000","last_became_active":"2024-02-14T17:41:58.041787+0000","last_became_peered":"2024-02-14T17:41:58.041787+0000","last_unstale":"2024-02-14T17:42:39.353857+0000","last_undegraded":"2024-02-14T17:42:39.353857+0000","last_fullsized":"2024-02-14T17:42:39.353857+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:58:43.706000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353533+0000","last_change":"2024-02-14T17:42:00.050014+0000","last_active":"2024-02-14T17:42:39.353533+0000","last_peered":"2024-02-14T17:42:39.353533+0000","last_clean":"2024-02-14T17:42:39.353533+0000","last_became_active":"2024-02-14T17:42:00.049653+0000","last_became_peered":"2024-02-14T17:42:00.049653+0000","last_unstale":"2024-02-14T17:42:39.353533+0000","last_undegraded":"2024-02-14T17:42:39.353533+0000","last_fullsized":"2024-02-14T17:42:39.353533+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:51:25.308096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353185+0000","last_change":"2024-02-14T17:42:04.075850+0000","last_active":"2024-02-14T17:42:39.353185+0000","last_peered":"2024-02-14T17:42:39.353185+0000","last_clean":"2024-02-14T17:42:39.353185+0000","last_became_active":"2024-02-14T17:42:04.075600+0000","last_became_peered":"2024-02-14T17:42:04.075600+0000","last_unstale":"2024-02-14T17:42:39.353185+0000","last_undegraded":"2024-02-14T17:42:39.353185+0000","last_fullsized":"2024-02-14T17:42:39.353185+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:36:11.838782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304196+0000","last_change":"2024-02-14T17:42:02.073943+0000","last_active":"2024-02-14T17:42:39.304196+0000","last_peered":"2024-02-14T17:42:39.304196+0000","last_clean":"2024-02-14T17:42:39.304196+0000","last_became_active":"2024-02-14T17:42:02.073635+0000","last_became_peered":"2024-02-14T17:42:02.073635+0000","last_unstale":"2024-02-14T17:42:39.304196+0000","last_undegraded":"2024-02-14T17:42:39.304196+0000","last_fullsized":"2024-02-14T17:42:39.304196+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:55:30.133503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353728+0000","last_change":"2024-02-14T17:41:58.042442+0000","last_active":"2024-02-14T17:42:39.353728+0000","last_peered":"2024-02-14T17:42:39.353728+0000","last_clean":"2024-02-14T17:42:39.353728+0000","last_became_active":"2024-02-14T17:41:58.042026+0000","last_became_peered":"2024-02-14T17:41:58.042026+0000","last_unstale":"2024-02-14T17:42:39.353728+0000","last_undegraded":"2024-02-14T17:42:39.353728+0000","last_fullsized":"2024-02-14T17:42:39.353728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:38:24.035282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"70'5","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354021+0000","last_change":"2024-02-14T17:42:00.050443+0000","last_active":"2024-02-14T17:42:39.354021+0000","last_peered":"2024-02-14T17:42:39.354021+0000","last_clean":"2024-02-14T17:42:39.354021+0000","last_became_active":"2024-02-14T17:42:00.049854+0000","last_became_peered":"2024-02-14T17:42:00.049854+0000","last_unstale":"2024-02-14T17:42:39.354021+0000","last_undegraded":"2024-02-14T17:42:39.354021+0000","last_fullsized":"2024-02-14T17:42:39.354021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:12:17.646365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354583+0000","last_change":"2024-02-14T17:42:04.355188+0000","last_active":"2024-02-14T17:42:39.354583+0000","last_peered":"2024-02-14T17:42:39.354583+0000","last_clean":"2024-02-14T17:42:39.354583+0000","last_became_active":"2024-02-14T17:42:04.354880+0000","last_became_peered":"2024-02-14T17:42:04.354880+0000","last_unstale":"2024-02-14T17:42:39.354583+0000","last_undegraded":"2024-02-14T17:42:39.354583+0000","last_fullsized":"2024-02-14T17:42:39.354583+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:51:09.492406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355079+0000","last_change":"2024-02-14T17:42:02.073499+0000","last_active":"2024-02-14T17:42:39.355079+0000","last_peered":"2024-02-14T17:42:39.355079+0000","last_clean":"2024-02-14T17:42:39.355079+0000","last_became_active":"2024-02-14T17:42:02.073009+0000","last_became_peered":"2024-02-14T17:42:02.073009+0000","last_unstale":"2024-02-14T17:42:39.355079+0000","last_undegraded":"2024-02-14T17:42:39.355079+0000","last_fullsized":"2024-02-14T17:42:39.355079+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:01:34.577460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302092+0000","last_change":"2024-02-14T17:41:58.032401+0000","last_active":"2024-02-14T17:42:39.302092+0000","last_peered":"2024-02-14T17:42:39.302092+0000","last_clean":"2024-02-14T17:42:39.302092+0000","last_became_active":"2024-02-14T17:41:58.032122+0000","last_became_peered":"2024-02-14T17:41:58.032122+0000","last_unstale":"2024-02-14T17:42:39.302092+0000","last_undegraded":"2024-02-14T17:42:39.302092+0000","last_fullsized":"2024-02-14T17:42:39.302092+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:53:00.320513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353791+0000","last_change":"2024-02-14T17:42:00.052767+0000","last_active":"2024-02-14T17:42:39.353791+0000","last_peered":"2024-02-14T17:42:39.353791+0000","last_clean":"2024-02-14T17:42:39.353791+0000","last_became_active":"2024-02-14T17:42:00.052424+0000","last_became_peered":"2024-02-14T17:42:00.052424+0000","last_unstale":"2024-02-14T17:42:39.353791+0000","last_undegraded":"2024-02-14T17:42:39.353791+0000","last_fullsized":"2024-02-14T17:42:39.353791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:53:01.074045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354796+0000","last_change":"2024-02-14T17:42:04.356240+0000","last_active":"2024-02-14T17:42:39.354796+0000","last_peered":"2024-02-14T17:42:39.354796+0000","last_clean":"2024-02-14T17:42:39.354796+0000","last_became_active":"2024-02-14T17:42:04.355591+0000","last_became_peered":"2024-02-14T17:42:04.355591+0000","last_unstale":"2024-02-14T17:42:39.354796+0000","last_undegraded":"2024-02-14T17:42:39.354796+0000","last_fullsized":"2024-02-14T17:42:39.354796+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:38:49.229772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354858+0000","last_change":"2024-02-14T17:42:02.067639+0000","last_active":"2024-02-14T17:42:39.354858+0000","last_peered":"2024-02-14T17:42:39.354858+0000","last_clean":"2024-02-14T17:42:39.354858+0000","last_became_active":"2024-02-14T17:42:02.067354+0000","last_became_peered":"2024-02-14T17:42:02.067354+0000","last_unstale":"2024-02-14T17:42:39.354858+0000","last_undegraded":"2024-02-14T17:42:39.354858+0000","last_fullsized":"2024-02-14T17:42:39.354858+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:42:45.684379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353659+0000","last_change":"2024-02-14T17:41:58.156267+0000","last_active":"2024-02-14T17:42:39.353659+0000","last_peered":"2024-02-14T17:42:39.353659+0000","last_clean":"2024-02-14T17:42:39.353659+0000","last_became_active":"2024-02-14T17:41:58.155909+0000","last_became_peered":"2024-02-14T17:41:58.155909+0000","last_unstale":"2024-02-14T17:42:39.353659+0000","last_undegraded":"2024-02-14T17:42:39.353659+0000","last_fullsized":"2024-02-14T17:42:39.353659+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:17:21.006331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"70'12","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354614+0000","last_change":"2024-02-14T17:42:21.971331+0000","last_active":"2024-02-14T17:42:39.354614+0000","last_peered":"2024-02-14T17:42:39.354614+0000","last_clean":"2024-02-14T17:42:39.354614+0000","last_became_active":"2024-02-14T17:42:21.965142+0000","last_became_peered":"2024-02-14T17:42:21.965142+0000","last_unstale":"2024-02-14T17:42:39.354614+0000","last_undegraded":"2024-02-14T17:42:39.354614+0000","last_fullsized":"2024-02-14T17:42:39.354614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:53:27.259376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305791+0000","last_change":"2024-02-14T17:42:04.074529+0000","last_active":"2024-02-14T17:42:39.305791+0000","last_peered":"2024-02-14T17:42:39.305791+0000","last_clean":"2024-02-14T17:42:39.305791+0000","last_became_active":"2024-02-14T17:42:04.074140+0000","last_became_peered":"2024-02-14T17:42:04.074140+0000","last_unstale":"2024-02-14T17:42:39.305791+0000","last_undegraded":"2024-02-14T17:42:39.305791+0000","last_fullsized":"2024-02-14T17:42:39.305791+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:31:51.240899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"70'8","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:18.210743+0000","last_change":"2024-02-14T17:42:02.061550+0000","last_active":"2024-02-14T17:43:18.210743+0000","last_peered":"2024-02-14T17:43:18.210743+0000","last_clean":"2024-02-14T17:43:18.210743+0000","last_became_active":"2024-02-14T17:42:02.061241+0000","last_became_peered":"2024-02-14T17:42:02.061241+0000","last_unstale":"2024-02-14T17:43:18.210743+0000","last_undegraded":"2024-02-14T17:43:18.210743+0000","last_fullsized":"2024-02-14T17:43:18.210743+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:25:57.686521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"63'2","reported_seq":40,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354444+0000","last_change":"2024-02-14T17:41:58.033635+0000","last_active":"2024-02-14T17:42:39.354444+0000","last_peered":"2024-02-14T17:42:39.354444+0000","last_clean":"2024-02-14T17:42:39.354444+0000","last_became_active":"2024-02-14T17:41:58.033312+0000","last_became_peered":"2024-02-14T17:41:58.033312+0000","last_unstale":"2024-02-14T17:42:39.354444+0000","last_undegraded":"2024-02-14T17:42:39.354444+0000","last_fullsized":"2024-02-14T17:42:39.354444+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:27:03.220076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302598+0000","last_change":"2024-02-14T17:42:00.139996+0000","last_active":"2024-02-14T17:42:39.302598+0000","last_peered":"2024-02-14T17:42:39.302598+0000","last_clean":"2024-02-14T17:42:39.302598+0000","last_became_active":"2024-02-14T17:42:00.139681+0000","last_became_peered":"2024-02-14T17:42:00.139681+0000","last_unstale":"2024-02-14T17:42:39.302598+0000","last_undegraded":"2024-02-14T17:42:39.302598+0000","last_fullsized":"2024-02-14T17:42:39.302598+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:59:44.349762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354789+0000","last_change":"2024-02-14T17:42:04.071580+0000","last_active":"2024-02-14T17:42:39.354789+0000","last_peered":"2024-02-14T17:42:39.354789+0000","last_clean":"2024-02-14T17:42:39.354789+0000","last_became_active":"2024-02-14T17:42:04.071220+0000","last_became_peered":"2024-02-14T17:42:04.071220+0000","last_unstale":"2024-02-14T17:42:39.354789+0000","last_undegraded":"2024-02-14T17:42:39.354789+0000","last_fullsized":"2024-02-14T17:42:39.354789+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:21:07.253754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302936+0000","last_change":"2024-02-14T17:42:02.061685+0000","last_active":"2024-02-14T17:42:39.302936+0000","last_peered":"2024-02-14T17:42:39.302936+0000","last_clean":"2024-02-14T17:42:39.302936+0000","last_became_active":"2024-02-14T17:42:02.061361+0000","last_became_peered":"2024-02-14T17:42:02.061361+0000","last_unstale":"2024-02-14T17:42:39.302936+0000","last_undegraded":"2024-02-14T17:42:39.302936+0000","last_fullsized":"2024-02-14T17:42:39.302936+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:56:26.735679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353275+0000","last_change":"2024-02-14T17:41:58.031978+0000","last_active":"2024-02-14T17:42:39.353275+0000","last_peered":"2024-02-14T17:42:39.353275+0000","last_clean":"2024-02-14T17:42:39.353275+0000","last_became_active":"2024-02-14T17:41:58.031514+0000","last_became_peered":"2024-02-14T17:41:58.031514+0000","last_unstale":"2024-02-14T17:42:39.353275+0000","last_undegraded":"2024-02-14T17:42:39.353275+0000","last_fullsized":"2024-02-14T17:42:39.353275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:42:22.060395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303881+0000","last_change":"2024-02-14T17:42:00.053869+0000","last_active":"2024-02-14T17:42:39.303881+0000","last_peered":"2024-02-14T17:42:39.303881+0000","last_clean":"2024-02-14T17:42:39.303881+0000","last_became_active":"2024-02-14T17:42:00.053411+0000","last_became_peered":"2024-02-14T17:42:00.053411+0000","last_unstale":"2024-02-14T17:42:39.303881+0000","last_undegraded":"2024-02-14T17:42:39.303881+0000","last_fullsized":"2024-02-14T17:42:39.303881+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:37:56.338543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304144+0000","last_change":"2024-02-14T17:42:04.074218+0000","last_active":"2024-02-14T17:42:39.304144+0000","last_peered":"2024-02-14T17:42:39.304144+0000","last_clean":"2024-02-14T17:42:39.304144+0000","last_became_active":"2024-02-14T17:42:04.073887+0000","last_became_peered":"2024-02-14T17:42:04.073887+0000","last_unstale":"2024-02-14T17:42:39.304144+0000","last_undegraded":"2024-02-14T17:42:39.304144+0000","last_fullsized":"2024-02-14T17:42:39.304144+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:24:10.551620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302750+0000","last_change":"2024-02-14T17:42:02.063328+0000","last_active":"2024-02-14T17:42:39.302750+0000","last_peered":"2024-02-14T17:42:39.302750+0000","last_clean":"2024-02-14T17:42:39.302750+0000","last_became_active":"2024-02-14T17:42:02.063028+0000","last_became_peered":"2024-02-14T17:42:02.063028+0000","last_unstale":"2024-02-14T17:42:39.302750+0000","last_undegraded":"2024-02-14T17:42:39.302750+0000","last_fullsized":"2024-02-14T17:42:39.302750+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:50:20.781895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302026+0000","last_change":"2024-02-14T17:41:58.039088+0000","last_active":"2024-02-14T17:42:39.302026+0000","last_peered":"2024-02-14T17:42:39.302026+0000","last_clean":"2024-02-14T17:42:39.302026+0000","last_became_active":"2024-02-14T17:41:58.037943+0000","last_became_peered":"2024-02-14T17:41:58.037943+0000","last_unstale":"2024-02-14T17:42:39.302026+0000","last_undegraded":"2024-02-14T17:42:39.302026+0000","last_fullsized":"2024-02-14T17:42:39.302026+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:01:00.146322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"70'4","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353382+0000","last_change":"2024-02-14T17:42:00.054135+0000","last_active":"2024-02-14T17:42:39.353382+0000","last_peered":"2024-02-14T17:42:39.353382+0000","last_clean":"2024-02-14T17:42:39.353382+0000","last_became_active":"2024-02-14T17:42:00.053812+0000","last_became_peered":"2024-02-14T17:42:00.053812+0000","last_unstale":"2024-02-14T17:42:39.353382+0000","last_undegraded":"2024-02-14T17:42:39.353382+0000","last_fullsized":"2024-02-14T17:42:39.353382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:31:24.841120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354356+0000","last_change":"2024-02-14T17:42:04.355940+0000","last_active":"2024-02-14T17:42:39.354356+0000","last_peered":"2024-02-14T17:42:39.354356+0000","last_clean":"2024-02-14T17:42:39.354356+0000","last_became_active":"2024-02-14T17:42:04.355168+0000","last_became_peered":"2024-02-14T17:42:04.355168+0000","last_unstale":"2024-02-14T17:42:39.354356+0000","last_undegraded":"2024-02-14T17:42:39.354356+0000","last_fullsized":"2024-02-14T17:42:39.354356+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:24:32.385496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303778+0000","last_change":"2024-02-14T17:42:02.064688+0000","last_active":"2024-02-14T17:42:39.303778+0000","last_peered":"2024-02-14T17:42:39.303778+0000","last_clean":"2024-02-14T17:42:39.303778+0000","last_became_active":"2024-02-14T17:42:02.064392+0000","last_became_peered":"2024-02-14T17:42:02.064392+0000","last_unstale":"2024-02-14T17:42:39.303778+0000","last_undegraded":"2024-02-14T17:42:39.303778+0000","last_fullsized":"2024-02-14T17:42:39.303778+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:02:12.496777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354227+0000","last_change":"2024-02-14T17:41:58.043010+0000","last_active":"2024-02-14T17:42:39.354227+0000","last_peered":"2024-02-14T17:42:39.354227+0000","last_clean":"2024-02-14T17:42:39.354227+0000","last_became_active":"2024-02-14T17:41:58.042656+0000","last_became_peered":"2024-02-14T17:41:58.042656+0000","last_unstale":"2024-02-14T17:42:39.354227+0000","last_undegraded":"2024-02-14T17:42:39.354227+0000","last_fullsized":"2024-02-14T17:42:39.354227+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:00:59.476277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354287+0000","last_change":"2024-02-14T17:42:00.141163+0000","last_active":"2024-02-14T17:42:39.354287+0000","last_peered":"2024-02-14T17:42:39.354287+0000","last_clean":"2024-02-14T17:42:39.354287+0000","last_became_active":"2024-02-14T17:42:00.140851+0000","last_became_peered":"2024-02-14T17:42:00.140851+0000","last_unstale":"2024-02-14T17:42:39.354287+0000","last_undegraded":"2024-02-14T17:42:39.354287+0000","last_fullsized":"2024-02-14T17:42:39.354287+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:03:15.318523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.300969+0000","last_change":"2024-02-14T17:42:04.065732+0000","last_active":"2024-02-14T17:42:39.300969+0000","last_peered":"2024-02-14T17:42:39.300969+0000","last_clean":"2024-02-14T17:42:39.300969+0000","last_became_active":"2024-02-14T17:42:04.065376+0000","last_became_peered":"2024-02-14T17:42:04.065376+0000","last_unstale":"2024-02-14T17:42:39.300969+0000","last_undegraded":"2024-02-14T17:42:39.300969+0000","last_fullsized":"2024-02-14T17:42:39.300969+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:06:47.477195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355196+0000","last_change":"2024-02-14T17:42:02.068711+0000","last_active":"2024-02-14T17:42:39.355196+0000","last_peered":"2024-02-14T17:42:39.355196+0000","last_clean":"2024-02-14T17:42:39.355196+0000","last_became_active":"2024-02-14T17:42:02.068404+0000","last_became_peered":"2024-02-14T17:42:02.068404+0000","last_unstale":"2024-02-14T17:42:39.355196+0000","last_undegraded":"2024-02-14T17:42:39.355196+0000","last_fullsized":"2024-02-14T17:42:39.355196+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:53:52.215133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304866+0000","last_change":"2024-02-14T17:41:58.036293+0000","last_active":"2024-02-14T17:42:39.304866+0000","last_peered":"2024-02-14T17:42:39.304866+0000","last_clean":"2024-02-14T17:42:39.304866+0000","last_became_active":"2024-02-14T17:41:58.035951+0000","last_became_peered":"2024-02-14T17:41:58.035951+0000","last_unstale":"2024-02-14T17:42:39.304866+0000","last_undegraded":"2024-02-14T17:42:39.304866+0000","last_fullsized":"2024-02-14T17:42:39.304866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:18:52.020383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354340+0000","last_change":"2024-02-14T17:42:00.047243+0000","last_active":"2024-02-14T17:42:39.354340+0000","last_peered":"2024-02-14T17:42:39.354340+0000","last_clean":"2024-02-14T17:42:39.354340+0000","last_became_active":"2024-02-14T17:42:00.046438+0000","last_became_peered":"2024-02-14T17:42:00.046438+0000","last_unstale":"2024-02-14T17:42:39.354340+0000","last_undegraded":"2024-02-14T17:42:39.354340+0000","last_fullsized":"2024-02-14T17:42:39.354340+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:25:28.078296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353807+0000","last_change":"2024-02-14T17:42:04.080840+0000","last_active":"2024-02-14T17:42:39.353807+0000","last_peered":"2024-02-14T17:42:39.353807+0000","last_clean":"2024-02-14T17:42:39.353807+0000","last_became_active":"2024-02-14T17:42:04.080456+0000","last_became_peered":"2024-02-14T17:42:04.080456+0000","last_unstale":"2024-02-14T17:42:39.353807+0000","last_undegraded":"2024-02-14T17:42:39.353807+0000","last_fullsized":"2024-02-14T17:42:39.353807+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:50:43.817082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354126+0000","last_change":"2024-02-14T17:42:02.071624+0000","last_active":"2024-02-14T17:42:39.354126+0000","last_peered":"2024-02-14T17:42:39.354126+0000","last_clean":"2024-02-14T17:42:39.354126+0000","last_became_active":"2024-02-14T17:42:02.071279+0000","last_became_peered":"2024-02-14T17:42:02.071279+0000","last_unstale":"2024-02-14T17:42:39.354126+0000","last_undegraded":"2024-02-14T17:42:39.354126+0000","last_fullsized":"2024-02-14T17:42:39.354126+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:50:59.905322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354731+0000","last_change":"2024-02-14T17:41:58.039692+0000","last_active":"2024-02-14T17:42:39.354731+0000","last_peered":"2024-02-14T17:42:39.354731+0000","last_clean":"2024-02-14T17:42:39.354731+0000","last_became_active":"2024-02-14T17:41:58.039373+0000","last_became_peered":"2024-02-14T17:41:58.039373+0000","last_unstale":"2024-02-14T17:42:39.354731+0000","last_undegraded":"2024-02-14T17:42:39.354731+0000","last_fullsized":"2024-02-14T17:42:39.354731+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:18:05.014382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354096+0000","last_change":"2024-02-14T17:42:00.043896+0000","last_active":"2024-02-14T17:42:39.354096+0000","last_peered":"2024-02-14T17:42:39.354096+0000","last_clean":"2024-02-14T17:42:39.354096+0000","last_became_active":"2024-02-14T17:42:00.043505+0000","last_became_peered":"2024-02-14T17:42:00.043505+0000","last_unstale":"2024-02-14T17:42:39.354096+0000","last_undegraded":"2024-02-14T17:42:39.354096+0000","last_fullsized":"2024-02-14T17:42:39.354096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:26:03.458942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302248+0000","last_change":"2024-02-14T17:42:04.075674+0000","last_active":"2024-02-14T17:42:39.302248+0000","last_peered":"2024-02-14T17:42:39.302248+0000","last_clean":"2024-02-14T17:42:39.302248+0000","last_became_active":"2024-02-14T17:42:04.075391+0000","last_became_peered":"2024-02-14T17:42:04.075391+0000","last_unstale":"2024-02-14T17:42:39.302248+0000","last_undegraded":"2024-02-14T17:42:39.302248+0000","last_fullsized":"2024-02-14T17:42:39.302248+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:47:51.331635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"70'8","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:18.210637+0000","last_change":"2024-02-14T17:42:02.065603+0000","last_active":"2024-02-14T17:43:18.210637+0000","last_peered":"2024-02-14T17:43:18.210637+0000","last_clean":"2024-02-14T17:43:18.210637+0000","last_became_active":"2024-02-14T17:42:02.065271+0000","last_became_peered":"2024-02-14T17:42:02.065271+0000","last_unstale":"2024-02-14T17:43:18.210637+0000","last_undegraded":"2024-02-14T17:43:18.210637+0000","last_fullsized":"2024-02-14T17:43:18.210637+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:00:47.566817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304316+0000","last_change":"2024-02-14T17:41:58.157507+0000","last_active":"2024-02-14T17:42:39.304316+0000","last_peered":"2024-02-14T17:42:39.304316+0000","last_clean":"2024-02-14T17:42:39.304316+0000","last_became_active":"2024-02-14T17:41:58.157133+0000","last_became_peered":"2024-02-14T17:41:58.157133+0000","last_unstale":"2024-02-14T17:42:39.304316+0000","last_undegraded":"2024-02-14T17:42:39.304316+0000","last_fullsized":"2024-02-14T17:42:39.304316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:21:37.614580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354828+0000","last_change":"2024-02-14T17:42:00.139951+0000","last_active":"2024-02-14T17:42:39.354828+0000","last_peered":"2024-02-14T17:42:39.354828+0000","last_clean":"2024-02-14T17:42:39.354828+0000","last_became_active":"2024-02-14T17:42:00.139630+0000","last_became_peered":"2024-02-14T17:42:00.139630+0000","last_unstale":"2024-02-14T17:42:39.354828+0000","last_undegraded":"2024-02-14T17:42:39.354828+0000","last_fullsized":"2024-02-14T17:42:39.354828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:20:19.278922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354548+0000","last_change":"2024-02-14T17:42:04.074089+0000","last_active":"2024-02-14T17:42:39.354548+0000","last_peered":"2024-02-14T17:42:39.354548+0000","last_clean":"2024-02-14T17:42:39.354548+0000","last_became_active":"2024-02-14T17:42:04.073825+0000","last_became_peered":"2024-02-14T17:42:04.073825+0000","last_unstale":"2024-02-14T17:42:39.354548+0000","last_undegraded":"2024-02-14T17:42:39.354548+0000","last_fullsized":"2024-02-14T17:42:39.354548+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:51.672907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"70'8","reported_seq":51,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:18.210369+0000","last_change":"2024-02-14T17:42:02.070809+0000","last_active":"2024-02-14T17:43:18.210369+0000","last_peered":"2024-02-14T17:43:18.210369+0000","last_clean":"2024-02-14T17:43:18.210369+0000","last_became_active":"2024-02-14T17:42:02.070479+0000","last_became_peered":"2024-02-14T17:42:02.070479+0000","last_unstale":"2024-02-14T17:43:18.210369+0000","last_undegraded":"2024-02-14T17:43:18.210369+0000","last_fullsized":"2024-02-14T17:43:18.210369+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:24:01.484179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.301209+0000","last_change":"2024-02-14T17:41:58.031279+0000","last_active":"2024-02-14T17:42:39.301209+0000","last_peered":"2024-02-14T17:42:39.301209+0000","last_clean":"2024-02-14T17:42:39.301209+0000","last_became_active":"2024-02-14T17:41:58.030713+0000","last_became_peered":"2024-02-14T17:41:58.030713+0000","last_unstale":"2024-02-14T17:42:39.301209+0000","last_undegraded":"2024-02-14T17:42:39.301209+0000","last_fullsized":"2024-02-14T17:42:39.301209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:49:54.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354061+0000","last_change":"2024-02-14T17:42:00.140621+0000","last_active":"2024-02-14T17:42:39.354061+0000","last_peered":"2024-02-14T17:42:39.354061+0000","last_clean":"2024-02-14T17:42:39.354061+0000","last_became_active":"2024-02-14T17:42:00.140301+0000","last_became_peered":"2024-02-14T17:42:00.140301+0000","last_unstale":"2024-02-14T17:42:39.354061+0000","last_undegraded":"2024-02-14T17:42:39.354061+0000","last_fullsized":"2024-02-14T17:42:39.354061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:52:11.686584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354598+0000","last_change":"2024-02-14T17:42:04.077953+0000","last_active":"2024-02-14T17:42:39.354598+0000","last_peered":"2024-02-14T17:42:39.354598+0000","last_clean":"2024-02-14T17:42:39.354598+0000","last_became_active":"2024-02-14T17:42:04.077620+0000","last_became_peered":"2024-02-14T17:42:04.077620+0000","last_unstale":"2024-02-14T17:42:39.354598+0000","last_undegraded":"2024-02-14T17:42:39.354598+0000","last_fullsized":"2024-02-14T17:42:39.354598+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:02:01.105691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355026+0000","last_change":"2024-02-14T17:42:02.069810+0000","last_active":"2024-02-14T17:42:39.355026+0000","last_peered":"2024-02-14T17:42:39.355026+0000","last_clean":"2024-02-14T17:42:39.355026+0000","last_became_active":"2024-02-14T17:42:02.069477+0000","last_became_peered":"2024-02-14T17:42:02.069477+0000","last_unstale":"2024-02-14T17:42:39.355026+0000","last_undegraded":"2024-02-14T17:42:39.355026+0000","last_fullsized":"2024-02-14T17:42:39.355026+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:20:15.353432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303936+0000","last_change":"2024-02-14T17:41:58.037798+0000","last_active":"2024-02-14T17:42:39.303936+0000","last_peered":"2024-02-14T17:42:39.303936+0000","last_clean":"2024-02-14T17:42:39.303936+0000","last_became_active":"2024-02-14T17:41:58.036822+0000","last_became_peered":"2024-02-14T17:41:58.036822+0000","last_unstale":"2024-02-14T17:42:39.303936+0000","last_undegraded":"2024-02-14T17:42:39.303936+0000","last_fullsized":"2024-02-14T17:42:39.303936+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:22:39.318435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302733+0000","last_change":"2024-02-14T17:42:00.140533+0000","last_active":"2024-02-14T17:42:39.302733+0000","last_peered":"2024-02-14T17:42:39.302733+0000","last_clean":"2024-02-14T17:42:39.302733+0000","last_became_active":"2024-02-14T17:42:00.139838+0000","last_became_peered":"2024-02-14T17:42:00.139838+0000","last_unstale":"2024-02-14T17:42:39.302733+0000","last_undegraded":"2024-02-14T17:42:39.302733+0000","last_fullsized":"2024-02-14T17:42:39.302733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:41:44.356387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304773+0000","last_change":"2024-02-14T17:42:04.357114+0000","last_active":"2024-02-14T17:42:39.304773+0000","last_peered":"2024-02-14T17:42:39.304773+0000","last_clean":"2024-02-14T17:42:39.304773+0000","last_became_active":"2024-02-14T17:42:04.356810+0000","last_became_peered":"2024-02-14T17:42:04.356810+0000","last_unstale":"2024-02-14T17:42:39.304773+0000","last_undegraded":"2024-02-14T17:42:39.304773+0000","last_fullsized":"2024-02-14T17:42:39.304773+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:27:17.103899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354231+0000","last_change":"2024-02-14T17:42:02.065716+0000","last_active":"2024-02-14T17:42:39.354231+0000","last_peered":"2024-02-14T17:42:39.354231+0000","last_clean":"2024-02-14T17:42:39.354231+0000","last_became_active":"2024-02-14T17:42:02.065386+0000","last_became_peered":"2024-02-14T17:42:02.065386+0000","last_unstale":"2024-02-14T17:42:39.354231+0000","last_undegraded":"2024-02-14T17:42:39.354231+0000","last_fullsized":"2024-02-14T17:42:39.354231+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:10:42.570899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302783+0000","last_change":"2024-02-14T17:41:58.040928+0000","last_active":"2024-02-14T17:42:39.302783+0000","last_peered":"2024-02-14T17:42:39.302783+0000","last_clean":"2024-02-14T17:42:39.302783+0000","last_became_active":"2024-02-14T17:41:58.040492+0000","last_became_peered":"2024-02-14T17:41:58.040492+0000","last_unstale":"2024-02-14T17:42:39.302783+0000","last_undegraded":"2024-02-14T17:42:39.302783+0000","last_fullsized":"2024-02-14T17:42:39.302783+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:29:52.954744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"70'6","reported_seq":38,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354973+0000","last_change":"2024-02-14T17:42:00.140217+0000","last_active":"2024-02-14T17:42:39.354973+0000","last_peered":"2024-02-14T17:42:39.354973+0000","last_clean":"2024-02-14T17:42:39.354973+0000","last_became_active":"2024-02-14T17:42:00.139585+0000","last_became_peered":"2024-02-14T17:42:00.139585+0000","last_unstale":"2024-02-14T17:42:39.354973+0000","last_undegraded":"2024-02-14T17:42:39.354973+0000","last_fullsized":"2024-02-14T17:42:39.354973+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:08:40.627653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353575+0000","last_change":"2024-02-14T17:42:04.356523+0000","last_active":"2024-02-14T17:42:39.353575+0000","last_peered":"2024-02-14T17:42:39.353575+0000","last_clean":"2024-02-14T17:42:39.353575+0000","last_became_active":"2024-02-14T17:42:04.356140+0000","last_became_peered":"2024-02-14T17:42:04.356140+0000","last_unstale":"2024-02-14T17:42:39.353575+0000","last_undegraded":"2024-02-14T17:42:39.353575+0000","last_fullsized":"2024-02-14T17:42:39.353575+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:17:03.561175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"63'2","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305392+0000","last_change":"2024-02-14T17:41:58.036370+0000","last_active":"2024-02-14T17:42:39.305392+0000","last_peered":"2024-02-14T17:42:39.305392+0000","last_clean":"2024-02-14T17:42:39.305392+0000","last_became_active":"2024-02-14T17:41:58.036059+0000","last_became_peered":"2024-02-14T17:41:58.036059+0000","last_unstale":"2024-02-14T17:42:39.305392+0000","last_undegraded":"2024-02-14T17:42:39.305392+0000","last_fullsized":"2024-02-14T17:42:39.305392+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:36:52.863582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353201+0000","last_change":"2024-02-14T17:42:00.140170+0000","last_active":"2024-02-14T17:42:39.353201+0000","last_peered":"2024-02-14T17:42:39.353201+0000","last_clean":"2024-02-14T17:42:39.353201+0000","last_became_active":"2024-02-14T17:42:00.139810+0000","last_became_peered":"2024-02-14T17:42:00.139810+0000","last_unstale":"2024-02-14T17:42:39.353201+0000","last_undegraded":"2024-02-14T17:42:39.353201+0000","last_fullsized":"2024-02-14T17:42:39.353201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:37:30.207519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"70'8","reported_seq":51,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:18.210651+0000","last_change":"2024-02-14T17:42:02.065804+0000","last_active":"2024-02-14T17:43:18.210651+0000","last_peered":"2024-02-14T17:43:18.210651+0000","last_clean":"2024-02-14T17:43:18.210651+0000","last_became_active":"2024-02-14T17:42:02.065425+0000","last_became_peered":"2024-02-14T17:42:02.065425+0000","last_unstale":"2024-02-14T17:43:18.210651+0000","last_undegraded":"2024-02-14T17:43:18.210651+0000","last_fullsized":"2024-02-14T17:43:18.210651+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:47:43.109365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354462+0000","last_change":"2024-02-14T17:42:04.078791+0000","last_active":"2024-02-14T17:42:39.354462+0000","last_peered":"2024-02-14T17:42:39.354462+0000","last_clean":"2024-02-14T17:42:39.354462+0000","last_became_active":"2024-02-14T17:42:04.078457+0000","last_became_peered":"2024-02-14T17:42:04.078457+0000","last_unstale":"2024-02-14T17:42:39.354462+0000","last_undegraded":"2024-02-14T17:42:39.354462+0000","last_fullsized":"2024-02-14T17:42:39.354462+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:24:46.172169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305441+0000","last_change":"2024-02-14T17:41:58.037428+0000","last_active":"2024-02-14T17:42:39.305441+0000","last_peered":"2024-02-14T17:42:39.305441+0000","last_clean":"2024-02-14T17:42:39.305441+0000","last_became_active":"2024-02-14T17:41:58.037084+0000","last_became_peered":"2024-02-14T17:41:58.037084+0000","last_unstale":"2024-02-14T17:42:39.305441+0000","last_undegraded":"2024-02-14T17:42:39.305441+0000","last_fullsized":"2024-02-14T17:42:39.305441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:53:18.073343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353783+0000","last_change":"2024-02-14T17:42:00.050716+0000","last_active":"2024-02-14T17:42:39.353783+0000","last_peered":"2024-02-14T17:42:39.353783+0000","last_clean":"2024-02-14T17:42:39.353783+0000","last_became_active":"2024-02-14T17:42:00.050000+0000","last_became_peered":"2024-02-14T17:42:00.050000+0000","last_unstale":"2024-02-14T17:42:39.353783+0000","last_undegraded":"2024-02-14T17:42:39.353783+0000","last_fullsized":"2024-02-14T17:42:39.353783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:50:10.957245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353000+0000","last_change":"2024-02-14T17:42:02.071692+0000","last_active":"2024-02-14T17:42:39.353000+0000","last_peered":"2024-02-14T17:42:39.353000+0000","last_clean":"2024-02-14T17:42:39.353000+0000","last_became_active":"2024-02-14T17:42:02.071380+0000","last_became_peered":"2024-02-14T17:42:02.071380+0000","last_unstale":"2024-02-14T17:42:39.353000+0000","last_undegraded":"2024-02-14T17:42:39.353000+0000","last_fullsized":"2024-02-14T17:42:39.353000+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:05:16.575354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303513+0000","last_change":"2024-02-14T17:42:04.080478+0000","last_active":"2024-02-14T17:42:39.303513+0000","last_peered":"2024-02-14T17:42:39.303513+0000","last_clean":"2024-02-14T17:42:39.303513+0000","last_became_active":"2024-02-14T17:42:04.080222+0000","last_became_peered":"2024-02-14T17:42:04.080222+0000","last_unstale":"2024-02-14T17:42:39.303513+0000","last_undegraded":"2024-02-14T17:42:39.303513+0000","last_fullsized":"2024-02-14T17:42:39.303513+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:40:27.894028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353576+0000","last_change":"2024-02-14T17:42:02.068426+0000","last_active":"2024-02-14T17:42:39.353576+0000","last_peered":"2024-02-14T17:42:39.353576+0000","last_clean":"2024-02-14T17:42:39.353576+0000","last_became_active":"2024-02-14T17:42:02.068068+0000","last_became_peered":"2024-02-14T17:42:02.068068+0000","last_unstale":"2024-02-14T17:42:39.353576+0000","last_undegraded":"2024-02-14T17:42:39.353576+0000","last_fullsized":"2024-02-14T17:42:39.353576+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:03:04.410879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305042+0000","last_change":"2024-02-14T17:41:58.040255+0000","last_active":"2024-02-14T17:42:39.305042+0000","last_peered":"2024-02-14T17:42:39.305042+0000","last_clean":"2024-02-14T17:42:39.305042+0000","last_became_active":"2024-02-14T17:41:58.039893+0000","last_became_peered":"2024-02-14T17:41:58.039893+0000","last_unstale":"2024-02-14T17:42:39.305042+0000","last_undegraded":"2024-02-14T17:42:39.305042+0000","last_fullsized":"2024-02-14T17:42:39.305042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:09:13.100236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"70'19","reported_seq":86,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303302+0000","last_change":"2024-02-14T17:42:21.979074+0000","last_active":"2024-02-14T17:42:39.303302+0000","last_peered":"2024-02-14T17:42:39.303302+0000","last_clean":"2024-02-14T17:42:39.303302+0000","last_became_active":"2024-02-14T17:42:21.965453+0000","last_became_peered":"2024-02-14T17:42:21.965453+0000","last_unstale":"2024-02-14T17:42:39.303302+0000","last_undegraded":"2024-02-14T17:42:39.303302+0000","last_fullsized":"2024-02-14T17:42:39.303302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:06:16.744933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.300965+0000","last_change":"2024-02-14T17:42:04.070583+0000","last_active":"2024-02-14T17:42:39.300965+0000","last_peered":"2024-02-14T17:42:39.300965+0000","last_clean":"2024-02-14T17:42:39.300965+0000","last_became_active":"2024-02-14T17:42:04.070263+0000","last_became_peered":"2024-02-14T17:42:04.070263+0000","last_unstale":"2024-02-14T17:42:39.300965+0000","last_undegraded":"2024-02-14T17:42:39.300965+0000","last_fullsized":"2024-02-14T17:42:39.300965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:52:55.132341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353334+0000","last_change":"2024-02-14T17:42:02.074131+0000","last_active":"2024-02-14T17:42:39.353334+0000","last_peered":"2024-02-14T17:42:39.353334+0000","last_clean":"2024-02-14T17:42:39.353334+0000","last_became_active":"2024-02-14T17:42:02.073717+0000","last_became_peered":"2024-02-14T17:42:02.073717+0000","last_unstale":"2024-02-14T17:42:39.353334+0000","last_undegraded":"2024-02-14T17:42:39.353334+0000","last_fullsized":"2024-02-14T17:42:39.353334+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:20:32.123349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353736+0000","last_change":"2024-02-14T17:41:58.044532+0000","last_active":"2024-02-14T17:42:39.353736+0000","last_peered":"2024-02-14T17:42:39.353736+0000","last_clean":"2024-02-14T17:42:39.353736+0000","last_became_active":"2024-02-14T17:41:58.044186+0000","last_became_peered":"2024-02-14T17:41:58.044186+0000","last_unstale":"2024-02-14T17:42:39.353736+0000","last_undegraded":"2024-02-14T17:42:39.353736+0000","last_fullsized":"2024-02-14T17:42:39.353736+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:48:22.993568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.301926+0000","last_change":"2024-02-14T17:42:00.044476+0000","last_active":"2024-02-14T17:42:39.301926+0000","last_peered":"2024-02-14T17:42:39.301926+0000","last_clean":"2024-02-14T17:42:39.301926+0000","last_became_active":"2024-02-14T17:42:00.044171+0000","last_became_peered":"2024-02-14T17:42:00.044171+0000","last_unstale":"2024-02-14T17:42:39.301926+0000","last_undegraded":"2024-02-14T17:42:39.301926+0000","last_fullsized":"2024-02-14T17:42:39.301926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:58:11.573231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354740+0000","last_change":"2024-02-14T17:42:04.079226+0000","last_active":"2024-02-14T17:42:39.354740+0000","last_peered":"2024-02-14T17:42:39.354740+0000","last_clean":"2024-02-14T17:42:39.354740+0000","last_became_active":"2024-02-14T17:42:04.078880+0000","last_became_peered":"2024-02-14T17:42:04.078880+0000","last_unstale":"2024-02-14T17:42:39.354740+0000","last_undegraded":"2024-02-14T17:42:39.354740+0000","last_fullsized":"2024-02-14T17:42:39.354740+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:37:52.069229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353415+0000","last_change":"2024-02-14T17:42:01.058940+0000","last_active":"2024-02-14T17:42:39.353415+0000","last_peered":"2024-02-14T17:42:39.353415+0000","last_clean":"2024-02-14T17:42:39.353415+0000","last_became_active":"2024-02-14T17:41:57.028333+0000","last_became_peered":"2024-02-14T17:41:57.028333+0000","last_unstale":"2024-02-14T17:42:39.353415+0000","last_undegraded":"2024-02-14T17:42:39.353415+0000","last_fullsized":"2024-02-14T17:42:39.353415+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_clean_scrub_stamp":"2024-02-14T17:41:56.017577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:53:06.948547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001143158,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353947+0000","last_change":"2024-02-14T17:41:58.037715+0000","last_active":"2024-02-14T17:42:39.353947+0000","last_peered":"2024-02-14T17:42:39.353947+0000","last_clean":"2024-02-14T17:42:39.353947+0000","last_became_active":"2024-02-14T17:41:58.037381+0000","last_became_peered":"2024-02-14T17:41:58.037381+0000","last_unstale":"2024-02-14T17:42:39.353947+0000","last_undegraded":"2024-02-14T17:42:39.353947+0000","last_fullsized":"2024-02-14T17:42:39.353947+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:59:36.165191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"70'28","reported_seq":95,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:18.210360+0000","last_change":"2024-02-14T17:42:00.053760+0000","last_active":"2024-02-14T17:43:18.210360+0000","last_peered":"2024-02-14T17:43:18.210360+0000","last_clean":"2024-02-14T17:43:18.210360+0000","last_became_active":"2024-02-14T17:42:00.053450+0000","last_became_peered":"2024-02-14T17:42:00.053450+0000","last_unstale":"2024-02-14T17:43:18.210360+0000","last_undegraded":"2024-02-14T17:43:18.210360+0000","last_fullsized":"2024-02-14T17:43:18.210360+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:11:55.086243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.301634+0000","last_change":"2024-02-14T17:42:02.067151+0000","last_active":"2024-02-14T17:42:39.301634+0000","last_peered":"2024-02-14T17:42:39.301634+0000","last_clean":"2024-02-14T17:42:39.301634+0000","last_became_active":"2024-02-14T17:42:02.066280+0000","last_became_peered":"2024-02-14T17:42:02.066280+0000","last_unstale":"2024-02-14T17:42:39.301634+0000","last_undegraded":"2024-02-14T17:42:39.301634+0000","last_fullsized":"2024-02-14T17:42:39.301634+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:04:27.169218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305215+0000","last_change":"2024-02-14T17:42:04.355594+0000","last_active":"2024-02-14T17:42:39.305215+0000","last_peered":"2024-02-14T17:42:39.305215+0000","last_clean":"2024-02-14T17:42:39.305215+0000","last_became_active":"2024-02-14T17:42:04.355262+0000","last_became_peered":"2024-02-14T17:42:04.355262+0000","last_unstale":"2024-02-14T17:42:39.305215+0000","last_undegraded":"2024-02-14T17:42:39.305215+0000","last_fullsized":"2024-02-14T17:42:39.305215+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:20:49.322891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354538+0000","last_change":"2024-02-14T17:41:58.158080+0000","last_active":"2024-02-14T17:42:39.354538+0000","last_peered":"2024-02-14T17:42:39.354538+0000","last_clean":"2024-02-14T17:42:39.354538+0000","last_became_active":"2024-02-14T17:41:58.157305+0000","last_became_peered":"2024-02-14T17:41:58.157305+0000","last_unstale":"2024-02-14T17:42:39.354538+0000","last_undegraded":"2024-02-14T17:42:39.354538+0000","last_fullsized":"2024-02-14T17:42:39.354538+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:20:00.476900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353051+0000","last_change":"2024-02-14T17:42:01.053171+0000","last_active":"2024-02-14T17:42:39.353051+0000","last_peered":"2024-02-14T17:42:39.353051+0000","last_clean":"2024-02-14T17:42:39.353051+0000","last_became_active":"2024-02-14T17:41:57.029052+0000","last_became_peered":"2024-02-14T17:41:57.029052+0000","last_unstale":"2024-02-14T17:42:39.353051+0000","last_undegraded":"2024-02-14T17:42:39.353051+0000","last_fullsized":"2024-02-14T17:42:39.353051+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_clean_scrub_stamp":"2024-02-14T17:41:56.017577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:53:20.653665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058337500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"70'13","reported_seq":56,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353855+0000","last_change":"2024-02-14T17:42:00.048089+0000","last_active":"2024-02-14T17:42:39.353855+0000","last_peered":"2024-02-14T17:42:39.353855+0000","last_clean":"2024-02-14T17:42:39.353855+0000","last_became_active":"2024-02-14T17:42:00.047771+0000","last_became_peered":"2024-02-14T17:42:00.047771+0000","last_unstale":"2024-02-14T17:42:39.353855+0000","last_undegraded":"2024-02-14T17:42:39.353855+0000","last_fullsized":"2024-02-14T17:42:39.353855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:38:06.471250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353999+0000","last_change":"2024-02-14T17:42:02.066483+0000","last_active":"2024-02-14T17:42:39.353999+0000","last_peered":"2024-02-14T17:42:39.353999+0000","last_clean":"2024-02-14T17:42:39.353999+0000","last_became_active":"2024-02-14T17:42:02.066155+0000","last_became_peered":"2024-02-14T17:42:02.066155+0000","last_unstale":"2024-02-14T17:42:39.353999+0000","last_undegraded":"2024-02-14T17:42:39.353999+0000","last_fullsized":"2024-02-14T17:42:39.353999+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:20:17.137160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354446+0000","last_change":"2024-02-14T17:42:04.072898+0000","last_active":"2024-02-14T17:42:39.354446+0000","last_peered":"2024-02-14T17:42:39.354446+0000","last_clean":"2024-02-14T17:42:39.354446+0000","last_became_active":"2024-02-14T17:42:04.072661+0000","last_became_peered":"2024-02-14T17:42:04.072661+0000","last_unstale":"2024-02-14T17:42:39.354446+0000","last_undegraded":"2024-02-14T17:42:39.354446+0000","last_fullsized":"2024-02-14T17:42:39.354446+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:48:56.117293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'128","reported_seq":279,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:41.398732+0000","last_change":"2024-02-14T17:41:48.169790+0000","last_active":"2024-02-14T17:42:41.398732+0000","last_peered":"2024-02-14T17:42:41.398732+0000","last_clean":"2024-02-14T17:42:41.398732+0000","last_became_active":"2024-02-14T17:41:48.169398+0000","last_became_peered":"2024-02-14T17:41:48.169398+0000","last_unstale":"2024-02-14T17:42:41.398732+0000","last_undegraded":"2024-02-14T17:42:41.398732+0000","last_fullsized":"2024-02-14T17:42:41.398732+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:39:39.206620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:39:39.206620+0000","last_clean_scrub_stamp":"2024-02-14T17:39:39.206620+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:09:05.151527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354255+0000","last_change":"2024-02-14T17:41:58.038148+0000","last_active":"2024-02-14T17:42:39.354255+0000","last_peered":"2024-02-14T17:42:39.354255+0000","last_clean":"2024-02-14T17:42:39.354255+0000","last_became_active":"2024-02-14T17:41:58.037856+0000","last_became_peered":"2024-02-14T17:41:58.037856+0000","last_unstale":"2024-02-14T17:42:39.354255+0000","last_undegraded":"2024-02-14T17:42:39.354255+0000","last_fullsized":"2024-02-14T17:42:39.354255+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:45:31.726044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"70'16","reported_seq":69,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:18.210704+0000","last_change":"2024-02-14T17:42:00.140373+0000","last_active":"2024-02-14T17:43:18.210704+0000","last_peered":"2024-02-14T17:43:18.210704+0000","last_clean":"2024-02-14T17:43:18.210704+0000","last_became_active":"2024-02-14T17:42:00.140055+0000","last_became_peered":"2024-02-14T17:42:00.140055+0000","last_unstale":"2024-02-14T17:43:18.210704+0000","last_undegraded":"2024-02-14T17:43:18.210704+0000","last_fullsized":"2024-02-14T17:43:18.210704+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:54:59.678414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.305967+0000","last_change":"2024-02-14T17:42:02.070709+0000","last_active":"2024-02-14T17:42:39.305967+0000","last_peered":"2024-02-14T17:42:39.305967+0000","last_clean":"2024-02-14T17:42:39.305967+0000","last_became_active":"2024-02-14T17:42:02.070402+0000","last_became_peered":"2024-02-14T17:42:02.070402+0000","last_unstale":"2024-02-14T17:42:39.305967+0000","last_undegraded":"2024-02-14T17:42:39.305967+0000","last_fullsized":"2024-02-14T17:42:39.305967+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:22:09.231236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353555+0000","last_change":"2024-02-14T17:42:04.079131+0000","last_active":"2024-02-14T17:42:39.353555+0000","last_peered":"2024-02-14T17:42:39.353555+0000","last_clean":"2024-02-14T17:42:39.353555+0000","last_became_active":"2024-02-14T17:42:04.078748+0000","last_became_peered":"2024-02-14T17:42:04.078748+0000","last_unstale":"2024-02-14T17:42:39.353555+0000","last_undegraded":"2024-02-14T17:42:39.353555+0000","last_fullsized":"2024-02-14T17:42:39.353555+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:29:58.643121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.301158+0000","last_change":"2024-02-14T17:41:58.033841+0000","last_active":"2024-02-14T17:42:39.301158+0000","last_peered":"2024-02-14T17:42:39.301158+0000","last_clean":"2024-02-14T17:42:39.301158+0000","last_became_active":"2024-02-14T17:41:58.033567+0000","last_became_peered":"2024-02-14T17:41:58.033567+0000","last_unstale":"2024-02-14T17:42:39.301158+0000","last_undegraded":"2024-02-14T17:42:39.301158+0000","last_fullsized":"2024-02-14T17:42:39.301158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:17:07.841067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"71'5","reported_seq":125,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:23.922589+0000","last_change":"2024-02-14T17:42:01.059673+0000","last_active":"2024-02-14T17:43:23.922589+0000","last_peered":"2024-02-14T17:43:23.922589+0000","last_clean":"2024-02-14T17:43:23.922589+0000","last_became_active":"2024-02-14T17:41:57.028428+0000","last_became_peered":"2024-02-14T17:41:57.028428+0000","last_unstale":"2024-02-14T17:43:23.922589+0000","last_undegraded":"2024-02-14T17:43:23.922589+0000","last_fullsized":"2024-02-14T17:43:23.922589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:56.017577+0000","last_clean_scrub_stamp":"2024-02-14T17:41:56.017577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:13:54.954585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022872890000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":80,"num_read_kb":75,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"70'12","reported_seq":47,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302347+0000","last_change":"2024-02-14T17:42:00.045240+0000","last_active":"2024-02-14T17:42:39.302347+0000","last_peered":"2024-02-14T17:42:39.302347+0000","last_clean":"2024-02-14T17:42:39.302347+0000","last_became_active":"2024-02-14T17:42:00.044171+0000","last_became_peered":"2024-02-14T17:42:00.044171+0000","last_unstale":"2024-02-14T17:42:39.302347+0000","last_undegraded":"2024-02-14T17:42:39.302347+0000","last_fullsized":"2024-02-14T17:42:39.302347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:04:55.458971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354680+0000","last_change":"2024-02-14T17:42:02.071968+0000","last_active":"2024-02-14T17:42:39.354680+0000","last_peered":"2024-02-14T17:42:39.354680+0000","last_clean":"2024-02-14T17:42:39.354680+0000","last_became_active":"2024-02-14T17:42:02.071641+0000","last_became_peered":"2024-02-14T17:42:02.071641+0000","last_unstale":"2024-02-14T17:42:39.354680+0000","last_undegraded":"2024-02-14T17:42:39.354680+0000","last_fullsized":"2024-02-14T17:42:39.354680+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:45:37.042480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354276+0000","last_change":"2024-02-14T17:42:04.355235+0000","last_active":"2024-02-14T17:42:39.354276+0000","last_peered":"2024-02-14T17:42:39.354276+0000","last_clean":"2024-02-14T17:42:39.354276+0000","last_became_active":"2024-02-14T17:42:04.354767+0000","last_became_peered":"2024-02-14T17:42:04.354767+0000","last_unstale":"2024-02-14T17:42:39.354276+0000","last_undegraded":"2024-02-14T17:42:39.354276+0000","last_fullsized":"2024-02-14T17:42:39.354276+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:39:56.384013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353684+0000","last_change":"2024-02-14T17:41:58.156156+0000","last_active":"2024-02-14T17:42:39.353684+0000","last_peered":"2024-02-14T17:42:39.353684+0000","last_clean":"2024-02-14T17:42:39.353684+0000","last_became_active":"2024-02-14T17:41:58.155754+0000","last_became_peered":"2024-02-14T17:41:58.155754+0000","last_unstale":"2024-02-14T17:42:39.353684+0000","last_undegraded":"2024-02-14T17:42:39.353684+0000","last_fullsized":"2024-02-14T17:42:39.353684+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:03:47.328120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"70'19","reported_seq":65,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302177+0000","last_change":"2024-02-14T17:42:00.140276+0000","last_active":"2024-02-14T17:42:39.302177+0000","last_peered":"2024-02-14T17:42:39.302177+0000","last_clean":"2024-02-14T17:42:39.302177+0000","last_became_active":"2024-02-14T17:42:00.139558+0000","last_became_peered":"2024-02-14T17:42:00.139558+0000","last_unstale":"2024-02-14T17:42:39.302177+0000","last_undegraded":"2024-02-14T17:42:39.302177+0000","last_fullsized":"2024-02-14T17:42:39.302177+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:09:21.737718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303463+0000","last_change":"2024-02-14T17:42:02.063057+0000","last_active":"2024-02-14T17:42:39.303463+0000","last_peered":"2024-02-14T17:42:39.303463+0000","last_clean":"2024-02-14T17:42:39.303463+0000","last_became_active":"2024-02-14T17:42:02.062781+0000","last_became_peered":"2024-02-14T17:42:02.062781+0000","last_unstale":"2024-02-14T17:42:39.303463+0000","last_undegraded":"2024-02-14T17:42:39.303463+0000","last_fullsized":"2024-02-14T17:42:39.303463+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:00:56.815449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.300965+0000","last_change":"2024-02-14T17:42:04.080204+0000","last_active":"2024-02-14T17:42:39.300965+0000","last_peered":"2024-02-14T17:42:39.300965+0000","last_clean":"2024-02-14T17:42:39.300965+0000","last_became_active":"2024-02-14T17:42:04.079811+0000","last_became_peered":"2024-02-14T17:42:04.079811+0000","last_unstale":"2024-02-14T17:42:39.300965+0000","last_undegraded":"2024-02-14T17:42:39.300965+0000","last_fullsized":"2024-02-14T17:42:39.300965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:37:45.857138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353964+0000","last_change":"2024-02-14T17:41:58.155985+0000","last_active":"2024-02-14T17:42:39.353964+0000","last_peered":"2024-02-14T17:42:39.353964+0000","last_clean":"2024-02-14T17:42:39.353964+0000","last_became_active":"2024-02-14T17:41:58.155597+0000","last_became_peered":"2024-02-14T17:41:58.155597+0000","last_unstale":"2024-02-14T17:42:39.353964+0000","last_undegraded":"2024-02-14T17:42:39.353964+0000","last_fullsized":"2024-02-14T17:42:39.353964+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:08:02.355959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354218+0000","last_change":"2024-02-14T17:42:00.050062+0000","last_active":"2024-02-14T17:42:39.354218+0000","last_peered":"2024-02-14T17:42:39.354218+0000","last_clean":"2024-02-14T17:42:39.354218+0000","last_became_active":"2024-02-14T17:42:00.049028+0000","last_became_peered":"2024-02-14T17:42:00.049028+0000","last_unstale":"2024-02-14T17:42:39.354218+0000","last_undegraded":"2024-02-14T17:42:39.354218+0000","last_fullsized":"2024-02-14T17:42:39.354218+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:29:37.283830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"70'8","reported_seq":51,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:18.210361+0000","last_change":"2024-02-14T17:42:02.060560+0000","last_active":"2024-02-14T17:43:18.210361+0000","last_peered":"2024-02-14T17:43:18.210361+0000","last_clean":"2024-02-14T17:43:18.210361+0000","last_became_active":"2024-02-14T17:42:02.060273+0000","last_became_peered":"2024-02-14T17:42:02.060273+0000","last_unstale":"2024-02-14T17:43:18.210361+0000","last_undegraded":"2024-02-14T17:43:18.210361+0000","last_fullsized":"2024-02-14T17:43:18.210361+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:18:11.905872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.304537+0000","last_change":"2024-02-14T17:42:04.356528+0000","last_active":"2024-02-14T17:42:39.304537+0000","last_peered":"2024-02-14T17:42:39.304537+0000","last_clean":"2024-02-14T17:42:39.304537+0000","last_became_active":"2024-02-14T17:42:04.356098+0000","last_became_peered":"2024-02-14T17:42:04.356098+0000","last_unstale":"2024-02-14T17:42:39.304537+0000","last_undegraded":"2024-02-14T17:42:39.304537+0000","last_fullsized":"2024-02-14T17:42:39.304537+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:17:24.241704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.302489+0000","last_change":"2024-02-14T17:41:58.032968+0000","last_active":"2024-02-14T17:42:39.302489+0000","last_peered":"2024-02-14T17:42:39.302489+0000","last_clean":"2024-02-14T17:42:39.302489+0000","last_became_active":"2024-02-14T17:41:58.032684+0000","last_became_peered":"2024-02-14T17:41:58.032684+0000","last_unstale":"2024-02-14T17:42:39.302489+0000","last_undegraded":"2024-02-14T17:42:39.302489+0000","last_fullsized":"2024-02-14T17:42:39.302489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:47:59.156383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"70'14","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353643+0000","last_change":"2024-02-14T17:42:00.046358+0000","last_active":"2024-02-14T17:42:39.353643+0000","last_peered":"2024-02-14T17:42:39.353643+0000","last_clean":"2024-02-14T17:42:39.353643+0000","last_became_active":"2024-02-14T17:42:00.045634+0000","last_became_peered":"2024-02-14T17:42:00.045634+0000","last_unstale":"2024-02-14T17:42:39.353643+0000","last_undegraded":"2024-02-14T17:42:39.353643+0000","last_fullsized":"2024-02-14T17:42:39.353643+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:35:33.676299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"70'8","reported_seq":51,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:18.210360+0000","last_change":"2024-02-14T17:42:02.074810+0000","last_active":"2024-02-14T17:43:18.210360+0000","last_peered":"2024-02-14T17:43:18.210360+0000","last_clean":"2024-02-14T17:43:18.210360+0000","last_became_active":"2024-02-14T17:42:02.073089+0000","last_became_peered":"2024-02-14T17:42:02.073089+0000","last_unstale":"2024-02-14T17:43:18.210360+0000","last_undegraded":"2024-02-14T17:43:18.210360+0000","last_fullsized":"2024-02-14T17:43:18.210360+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:04:20.865557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353365+0000","last_change":"2024-02-14T17:42:04.079675+0000","last_active":"2024-02-14T17:42:39.353365+0000","last_peered":"2024-02-14T17:42:39.353365+0000","last_clean":"2024-02-14T17:42:39.353365+0000","last_became_active":"2024-02-14T17:42:04.079387+0000","last_became_peered":"2024-02-14T17:42:04.079387+0000","last_unstale":"2024-02-14T17:42:39.353365+0000","last_undegraded":"2024-02-14T17:42:39.353365+0000","last_fullsized":"2024-02-14T17:42:39.353365+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:01:31.759894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354841+0000","last_change":"2024-02-14T17:41:58.031823+0000","last_active":"2024-02-14T17:42:39.354841+0000","last_peered":"2024-02-14T17:42:39.354841+0000","last_clean":"2024-02-14T17:42:39.354841+0000","last_became_active":"2024-02-14T17:41:58.031463+0000","last_became_peered":"2024-02-14T17:41:58.031463+0000","last_unstale":"2024-02-14T17:42:39.354841+0000","last_undegraded":"2024-02-14T17:42:39.354841+0000","last_fullsized":"2024-02-14T17:42:39.354841+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:08:21.970694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"70'18","reported_seq":61,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353660+0000","last_change":"2024-02-14T17:42:00.141209+0000","last_active":"2024-02-14T17:42:39.353660+0000","last_peered":"2024-02-14T17:42:39.353660+0000","last_clean":"2024-02-14T17:42:39.353660+0000","last_became_active":"2024-02-14T17:42:00.140912+0000","last_became_peered":"2024-02-14T17:42:00.140912+0000","last_unstale":"2024-02-14T17:42:39.353660+0000","last_undegraded":"2024-02-14T17:42:39.353660+0000","last_fullsized":"2024-02-14T17:42:39.353660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:58:51.413490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353284+0000","last_change":"2024-02-14T17:42:02.066962+0000","last_active":"2024-02-14T17:42:39.353284+0000","last_peered":"2024-02-14T17:42:39.353284+0000","last_clean":"2024-02-14T17:42:39.353284+0000","last_became_active":"2024-02-14T17:42:02.065889+0000","last_became_peered":"2024-02-14T17:42:02.065889+0000","last_unstale":"2024-02-14T17:42:39.353284+0000","last_undegraded":"2024-02-14T17:42:39.353284+0000","last_fullsized":"2024-02-14T17:42:39.353284+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:09:12.526329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353790+0000","last_change":"2024-02-14T17:42:04.072382+0000","last_active":"2024-02-14T17:42:39.353790+0000","last_peered":"2024-02-14T17:42:39.353790+0000","last_clean":"2024-02-14T17:42:39.353790+0000","last_became_active":"2024-02-14T17:42:04.072126+0000","last_became_peered":"2024-02-14T17:42:04.072126+0000","last_unstale":"2024-02-14T17:42:39.353790+0000","last_undegraded":"2024-02-14T17:42:39.353790+0000","last_fullsized":"2024-02-14T17:42:39.353790+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:33:07.266337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"70'8","reported_seq":51,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:18.210637+0000","last_change":"2024-02-14T17:42:02.060502+0000","last_active":"2024-02-14T17:43:18.210637+0000","last_peered":"2024-02-14T17:43:18.210637+0000","last_clean":"2024-02-14T17:43:18.210637+0000","last_became_active":"2024-02-14T17:42:02.060196+0000","last_became_peered":"2024-02-14T17:42:02.060196+0000","last_unstale":"2024-02-14T17:43:18.210637+0000","last_undegraded":"2024-02-14T17:43:18.210637+0000","last_fullsized":"2024-02-14T17:43:18.210637+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:00:27.871131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353570+0000","last_change":"2024-02-14T17:41:58.036259+0000","last_active":"2024-02-14T17:42:39.353570+0000","last_peered":"2024-02-14T17:42:39.353570+0000","last_clean":"2024-02-14T17:42:39.353570+0000","last_became_active":"2024-02-14T17:41:58.035918+0000","last_became_peered":"2024-02-14T17:41:58.035918+0000","last_unstale":"2024-02-14T17:42:39.353570+0000","last_undegraded":"2024-02-14T17:42:39.353570+0000","last_fullsized":"2024-02-14T17:42:39.353570+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:12:06.124705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353945+0000","last_change":"2024-02-14T17:42:00.053171+0000","last_active":"2024-02-14T17:42:39.353945+0000","last_peered":"2024-02-14T17:42:39.353945+0000","last_clean":"2024-02-14T17:42:39.353945+0000","last_became_active":"2024-02-14T17:42:00.052841+0000","last_became_peered":"2024-02-14T17:42:00.052841+0000","last_unstale":"2024-02-14T17:42:39.353945+0000","last_undegraded":"2024-02-14T17:42:39.353945+0000","last_fullsized":"2024-02-14T17:42:39.353945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:54:39.633547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354582+0000","last_change":"2024-02-14T17:42:04.355576+0000","last_active":"2024-02-14T17:42:39.354582+0000","last_peered":"2024-02-14T17:42:39.354582+0000","last_clean":"2024-02-14T17:42:39.354582+0000","last_became_active":"2024-02-14T17:42:04.355227+0000","last_became_peered":"2024-02-14T17:42:04.355227+0000","last_unstale":"2024-02-14T17:42:39.354582+0000","last_undegraded":"2024-02-14T17:42:39.354582+0000","last_fullsized":"2024-02-14T17:42:39.354582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:29:37.481330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354671+0000","last_change":"2024-02-14T17:42:02.063035+0000","last_active":"2024-02-14T17:42:39.354671+0000","last_peered":"2024-02-14T17:42:39.354671+0000","last_clean":"2024-02-14T17:42:39.354671+0000","last_became_active":"2024-02-14T17:42:02.062653+0000","last_became_peered":"2024-02-14T17:42:02.062653+0000","last_unstale":"2024-02-14T17:42:39.354671+0000","last_undegraded":"2024-02-14T17:42:39.354671+0000","last_fullsized":"2024-02-14T17:42:39.354671+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:00:23.049275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353386+0000","last_change":"2024-02-14T17:41:58.156296+0000","last_active":"2024-02-14T17:42:39.353386+0000","last_peered":"2024-02-14T17:42:39.353386+0000","last_clean":"2024-02-14T17:42:39.353386+0000","last_became_active":"2024-02-14T17:41:58.155924+0000","last_became_peered":"2024-02-14T17:41:58.155924+0000","last_unstale":"2024-02-14T17:42:39.353386+0000","last_undegraded":"2024-02-14T17:42:39.353386+0000","last_fullsized":"2024-02-14T17:42:39.353386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:18:47.062335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354454+0000","last_change":"2024-02-14T17:42:00.043521+0000","last_active":"2024-02-14T17:42:39.354454+0000","last_peered":"2024-02-14T17:42:39.354454+0000","last_clean":"2024-02-14T17:42:39.354454+0000","last_became_active":"2024-02-14T17:42:00.043124+0000","last_became_peered":"2024-02-14T17:42:00.043124+0000","last_unstale":"2024-02-14T17:42:39.354454+0000","last_undegraded":"2024-02-14T17:42:39.354454+0000","last_fullsized":"2024-02-14T17:42:39.354454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:36:49.304574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.353613+0000","last_change":"2024-02-14T17:42:04.080634+0000","last_active":"2024-02-14T17:42:39.353613+0000","last_peered":"2024-02-14T17:42:39.353613+0000","last_clean":"2024-02-14T17:42:39.353613+0000","last_became_active":"2024-02-14T17:42:04.080127+0000","last_became_peered":"2024-02-14T17:42:04.080127+0000","last_unstale":"2024-02-14T17:42:39.353613+0000","last_undegraded":"2024-02-14T17:42:39.353613+0000","last_fullsized":"2024-02-14T17:42:39.353613+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:22:15.465265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354085+0000","last_change":"2024-02-14T17:42:02.072161+0000","last_active":"2024-02-14T17:42:39.354085+0000","last_peered":"2024-02-14T17:42:39.354085+0000","last_clean":"2024-02-14T17:42:39.354085+0000","last_became_active":"2024-02-14T17:42:02.071813+0000","last_became_peered":"2024-02-14T17:42:02.071813+0000","last_unstale":"2024-02-14T17:42:39.354085+0000","last_undegraded":"2024-02-14T17:42:39.354085+0000","last_fullsized":"2024-02-14T17:42:39.354085+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:27:07.653643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.303004+0000","last_change":"2024-02-14T17:41:58.032975+0000","last_active":"2024-02-14T17:42:39.303004+0000","last_peered":"2024-02-14T17:42:39.303004+0000","last_clean":"2024-02-14T17:42:39.303004+0000","last_became_active":"2024-02-14T17:41:58.032623+0000","last_became_peered":"2024-02-14T17:41:58.032623+0000","last_unstale":"2024-02-14T17:42:39.303004+0000","last_undegraded":"2024-02-14T17:42:39.303004+0000","last_fullsized":"2024-02-14T17:42:39.303004+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:36:56.678002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"70'17","reported_seq":57,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354365+0000","last_change":"2024-02-14T17:42:00.046141+0000","last_active":"2024-02-14T17:42:39.354365+0000","last_peered":"2024-02-14T17:42:39.354365+0000","last_clean":"2024-02-14T17:42:39.354365+0000","last_became_active":"2024-02-14T17:42:00.045716+0000","last_became_peered":"2024-02-14T17:42:00.045716+0000","last_unstale":"2024-02-14T17:42:39.354365+0000","last_undegraded":"2024-02-14T17:42:39.354365+0000","last_fullsized":"2024-02-14T17:42:39.354365+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:56:05.037366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354772+0000","last_change":"2024-02-14T17:42:04.074080+0000","last_active":"2024-02-14T17:42:39.354772+0000","last_peered":"2024-02-14T17:42:39.354772+0000","last_clean":"2024-02-14T17:42:39.354772+0000","last_became_active":"2024-02-14T17:42:04.073799+0000","last_became_peered":"2024-02-14T17:42:04.073799+0000","last_unstale":"2024-02-14T17:42:39.354772+0000","last_undegraded":"2024-02-14T17:42:39.354772+0000","last_fullsized":"2024-02-14T17:42:39.354772+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:03.053088+0000","last_clean_scrub_stamp":"2024-02-14T17:42:03.053088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:50:38.076407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"70'8","reported_seq":51,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:43:18.210426+0000","last_change":"2024-02-14T17:42:02.065914+0000","last_active":"2024-02-14T17:43:18.210426+0000","last_peered":"2024-02-14T17:43:18.210426+0000","last_clean":"2024-02-14T17:43:18.210426+0000","last_became_active":"2024-02-14T17:42:02.065569+0000","last_became_peered":"2024-02-14T17:42:02.065569+0000","last_unstale":"2024-02-14T17:43:18.210426+0000","last_undegraded":"2024-02-14T17:43:18.210426+0000","last_fullsized":"2024-02-14T17:43:18.210426+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:01.046084+0000","last_clean_scrub_stamp":"2024-02-14T17:42:01.046084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:16:00.343907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.355251+0000","last_change":"2024-02-14T17:41:58.033631+0000","last_active":"2024-02-14T17:42:39.355251+0000","last_peered":"2024-02-14T17:42:39.355251+0000","last_clean":"2024-02-14T17:42:39.355251+0000","last_became_active":"2024-02-14T17:41:58.033337+0000","last_became_peered":"2024-02-14T17:41:58.033337+0000","last_unstale":"2024-02-14T17:42:39.355251+0000","last_undegraded":"2024-02-14T17:42:39.355251+0000","last_fullsized":"2024-02-14T17:42:39.355251+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:57.018784+0000","last_clean_scrub_stamp":"2024-02-14T17:41:57.018784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:31:09.656319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-14T17:42:39.354548+0000","last_change":"2024-02-14T17:42:00.050280+0000","last_active":"2024-02-14T17:42:39.354548+0000","last_peered":"2024-02-14T17:42:39.354548+0000","last_clean":"2024-02-14T17:42:39.354548+0000","last_became_active":"2024-02-14T17:42:00.049856+0000","last_became_peered":"2024-02-14T17:42:00.049856+0000","last_unstale":"2024-02-14T17:42:39.354548+0000","last_undegraded":"2024-02-14T17:42:39.354548+0000","last_fullsized":"2024-02-14T17:42:39.354548+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:41:59.028975+0000","last_clean_scrub_stamp":"2024-02-14T17:41:59.028975+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:42:39.563143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":75,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168599,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1331200,"data_stored":879723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63500000000000001}]},{"osd":1,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51800000000000002}]},{"osd":2,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60199999999999998}]},{"osd":3,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.66000000000000003}]},{"osd":4,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.67900000000000005}]},{"osd":6,"last update":"Wed Feb 14 17:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.64900000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462938,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28300,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716852,"statfs":{"total":95995035648,"available":95966056448,"internally_reserved":0,"allocated":1314816,"data_stored":878685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.38300000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66300000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68700000000000006,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Feb 14 17:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52000000000000002,"15min":0.51500000000000001},"min":{"1min":0.35799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.35999999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72499999999999998,"5min":0.754,"15min":0.754},"last":0.57199999999999995}]},{"osd":2,"last update":"Wed Feb 14 17:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.55500000000000005}]},{"osd":3,"last update":"Wed Feb 14 17:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.39400000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.38200000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed Feb 14 17:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.39300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.378},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.42399999999999999,"15min":0.41599999999999998},"min":{"1min":0.30099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.41899999999999998}]},{"osd":5,"last update":"Wed Feb 14 17:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.441,"15min":0.432},"min":{"1min":0.32900000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.378,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61699999999999999}]},{"osd":7,"last update":"Wed Feb 14 17:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.66700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31928,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713224,"statfs":{"total":95995035648,"available":95962341376,"internally_reserved":0,"allocated":765952,"data_stored":289173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:42:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33700000000000002,"15min":0.315},"min":{"1min":0.24299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.254},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.34599999999999997,"15min":0.315},"min":{"1min":0.29699999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.37}]},{"osd":2,"last update":"Wed Feb 14 17:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.39000000000000001,"15min":0.37},"min":{"1min":0.248,"5min":0.221,"15min":0.221},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.441,"15min":0.41499999999999998},"min":{"1min":0.34699999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.34599999999999997}]},{"osd":3,"last update":"Wed Feb 14 17:43:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.439,"15min":0.435},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.308},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.434,"15min":0.436},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.32100000000000001}]},{"osd":4,"last update":"Wed Feb 14 17:42:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.47799999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.65900000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.621},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed Feb 14 17:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.0469999999999997,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed Feb 14 17:42:22 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.498},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Wed Feb 14 17:42:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.0110000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.65400000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1331200,"data_stored":878592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 17:43:20 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.46400000000000002,"15min":0.55800000000000005},"min":{"1min":0.23100000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.54600000000000004,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.38500000000000001,"15min":0.35499999999999998},"min":{"1min":0.254,"5min":0.187,"15min":0.187},"max":{"1min":0.58999999999999997,"5min":0.626,"15min":0.626},"last":0.37}]},{"osd":2,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41899999999999998,"15min":0.40899999999999997},"min":{"1min":0.28599999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54500000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.252},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.39000000000000001,"15min":0.38300000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.315}]},{"osd":3,"last update":"Wed Feb 14 17:43:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.47699999999999998,"15min":0.49199999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65100000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.52300000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.35199999999999998}]},{"osd":4,"last update":"Wed Feb 14 17:42:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.621,"15min":0.629},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80400000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.60099999999999998,"15min":0.60799999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82199999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.59299999999999997,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.67800000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.628,"15min":0.64000000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71199999999999997,"5min":0.876,"15min":0.876},"last":0.61499999999999999}]},{"osd":6,"last update":"Wed Feb 14 17:42:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.629}]},{"osd":7,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.54600000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31884,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713268,"statfs":{"total":95995035648,"available":95962386432,"internally_reserved":0,"allocated":720896,"data_stored":287874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40100000000000002,"15min":0.39800000000000002},"min":{"1min":0.27700000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.308}]},{"osd":1,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41799999999999998,"15min":0.41599999999999998},"min":{"1min":0.26300000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.34599999999999997}]},{"osd":3,"last update":"Wed Feb 14 17:43:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44500000000000001,"15min":0.45300000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.32100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67800000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Feb 14 17:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53800000000000003}]},{"osd":5,"last update":"Wed Feb 14 17:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.80200000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed Feb 14 17:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.437}]},{"osd":7,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.55000000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31920,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713232,"statfs":{"total":95995035648,"available":95962349568,"internally_reserved":0,"allocated":757760,"data_stored":288732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:43:06 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.378,"15min":0.378},"min":{"1min":0.23899999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.499,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.437,"15min":0.439},"min":{"1min":0.26500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.63200000000000001,"5min":0.746,"15min":0.746},"last":0.32100000000000001}]},{"osd":1,"last update":"Wed Feb 14 17:43:06 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.41899999999999998,"15min":0.42399999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.59399999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.36799999999999999}]},{"osd":2,"last update":"Wed Feb 14 17:43:06 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46400000000000002,"15min":0.46200000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66500000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.46200000000000002,"15min":0.46800000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.33700000000000002}]},{"osd":4,"last update":"Wed Feb 14 17:42:31 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77700000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.42999999999999999}]},{"osd":5,"last update":"Wed Feb 14 17:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78600000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.498},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77600000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Feb 14 17:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79000000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.625,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.71599999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46600000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986214,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31908,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713244,"statfs":{"total":95995035648,"available":95962361856,"internally_reserved":0,"allocated":745472,"data_stored":288433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.40200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed Feb 14 17:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51700000000000002,"15min":0.51600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70899999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.66500000000000004}]},{"osd":2,"last update":"Wed Feb 14 17:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65200000000000002}]},{"osd":3,"last update":"Wed Feb 14 17:42:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.42199999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.72099999999999997}]},{"osd":5,"last update":"Wed Feb 14 17:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.29199999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.67200000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.39000000000000001,"15min":0.379},"min":{"1min":0.28699999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61699999999999999}]},{"osd":6,"last update":"Wed Feb 14 17:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.46200000000000002,"15min":0.44800000000000001},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.46200000000000002,"15min":0.442},"min":{"1min":0.36599999999999999,"5min":0.315,"15min":0.315},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Wed Feb 14 17:42:44 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.79800000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724576,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27732,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717420,"statfs":{"total":95995035648,"available":95966638080,"internally_reserved":0,"allocated":729088,"data_stored":288752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.61499999999999999}]},{"osd":1,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76500000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.69499999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.42299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.67300000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.68799999999999994}]},{"osd":3,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.65000000000000002}]},{"osd":4,"last update":"Wed Feb 14 17:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39000000000000001,"15min":0.38200000000000001},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.39200000000000002,"15min":0.378},"min":{"1min":0.33700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45500000000000002}]},{"osd":6,"last update":"Wed Feb 14 17:43:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.40699999999999997,"15min":0.39700000000000002},"min":{"1min":0.308,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.439,"15min":0.42699999999999999},"min":{"1min":0.317,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71199999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.48299999999999998}]},{"osd":7,"last update":"Wed Feb 14 17:42:46 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.52200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T17:43:28.309 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-14T17:43:28.309 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-14T17:43:28.310 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-14T17:43:28.310 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 -- ceph health --format=json 2024-02-14T17:43:28.550 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:43:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:28 smithi184 ceph-mon[33536]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:28 smithi184 ceph-mon[33536]: from='client.34643 v1:172.21.15.178:0/778231243' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:43:28.983 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:28 smithi178 ceph-mon[39811]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:28.983 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:28 smithi178 ceph-mon[39811]: from='client.34643 v1:172.21.15.178:0/778231243' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:43:28.983 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:28 smithi178 ceph-mon[30321]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:28.983 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:28 smithi178 ceph-mon[30321]: from='client.34643 v1:172.21.15.178:0/778231243' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:43:30.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.426+0000 7f979007a700 1 -- 172.21.15.178:0/838647195 >> v1:172.21.15.178:6790/0 conn(0x7f97881097b0 legacy=0x7f978810bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:30.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.426+0000 7f979007a700 1 -- 172.21.15.178:0/838647195 shutdown_connections 2024-02-14T17:43:30.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.426+0000 7f979007a700 1 -- 172.21.15.178:0/838647195 >> 172.21.15.178:0/838647195 conn(0x7f97880fc410 msgr2=0x7f97880fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:30.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.426+0000 7f979007a700 1 -- 172.21.15.178:0/838647195 shutdown_connections 2024-02-14T17:43:30.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.426+0000 7f979007a700 1 -- 172.21.15.178:0/838647195 wait complete. 2024-02-14T17:43:30.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f979007a700 1 Processor -- start 2024-02-14T17:43:30.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f979007a700 1 -- start start 2024-02-14T17:43:30.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f979007a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f97881a6890 con 0x7f97881097b0 2024-02-14T17:43:30.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f979007a700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f97881a7a70 con 0x7f9788106900 2024-02-14T17:43:30.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f979007a700 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f97881a8c70 con 0x7f978810d380 2024-02-14T17:43:30.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f978d615700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f97881097b0 0x7f97881a1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:46390/0 (socket says 172.21.15.178:46390) 2024-02-14T17:43:30.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f978de16700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f9788106900 0x7f978810ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.178:39378/0 (socket says 172.21.15.178:39378) 2024-02-14T17:43:30.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f978d615700 1 -- 172.21.15.178:0/1447999510 learned_addr learned my addr 172.21.15.178:0/1447999510 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:43:30.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937570492 0 0) 0x7f97881a6890 con 0x7f97881097b0 2024-02-14T17:43:30.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97700031f0 con 0x7f97881097b0 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883669166 0 0) 0x7f97881a8c70 con 0x7f978810d380 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97881a6890 con 0x7f978810d380 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471915404 0 0) 0x7f97881a7a70 con 0x7f9788106900 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97881a8c70 con 0x7f9788106900 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2533060160 0 0) 0x7f97700031f0 con 0x7f97881097b0 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f97881a7a70 con 0x7f97881097b0 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f977c004410 con 0x7f97881097b0 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2716985453 0 0) 0x7f97881a8c70 con 0x7f9788106900 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f97700031f0 con 0x7f9788106900 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.427+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2135608671 0 0) 0x7f97881a6890 con 0x7f978810d380 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.428+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f97881a8c70 con 0x7f978810d380 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.428+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f9780001af0 con 0x7f9788106900 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.428+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f9778001850 con 0x7f978810d380 2024-02-14T17:43:30.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.428+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2568372773 0 0) 0x7f97881a7a70 con 0x7f97881097b0 2024-02-14T17:43:30.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.428+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 >> v1:172.21.15.178:6790/0 conn(0x7f978810d380 legacy=0x7f97881a4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:30.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.428+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 >> v1:172.21.15.184:6789/0 conn(0x7f9788106900 legacy=0x7f978810ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:30.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.428+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97881a9e70 con 0x7f97881097b0 2024-02-14T17:43:30.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.428+0000 7f979007a700 1 -- 172.21.15.178:0/1447999510 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97881a8e60 con 0x7f97881097b0 2024-02-14T17:43:30.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.428+0000 7f979007a700 1 -- 172.21.15.178:0/1447999510 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f97881a92e0 con 0x7f97881097b0 2024-02-14T17:43:30.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.428+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f977c004570 con 0x7f97881097b0 2024-02-14T17:43:30.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.428+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f977c004d80 con 0x7f97881097b0 2024-02-14T17:43:30.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.429+0000 7f979007a700 1 -- 172.21.15.178:0/1447999510 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9774005120 con 0x7f97881097b0 2024-02-14T17:43:30.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.429+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f977c005030 con 0x7f97881097b0 2024-02-14T17:43:30.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.430+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 6259+0+0 (unknown 1818281892 0 0) 0x7f977c088a50 con 0x7f97881097b0 2024-02-14T17:43:30.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.434+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f977c055cf0 con 0x7f97881097b0 2024-02-14T17:43:30.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:30 smithi178 ceph-mon[30321]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:30.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:30 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:30.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:30 smithi178 ceph-mon[39811]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:30.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:30 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:30.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.671+0000 7f979007a700 1 -- 172.21.15.178:0/1447999510 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9774005950 con 0x7f97881097b0 2024-02-14T17:43:30.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.671+0000 7f9786ffd700 1 -- 172.21.15.178:0/1447999510 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7f977c054060 con 0x7f97881097b0 2024-02-14T17:43:30.673 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-02-14T17:43:30.673 INFO:teuthology.orchestra.run.smithi178.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-14T17:43:30.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.674+0000 7f979007a700 1 -- 172.21.15.178:0/1447999510 >> v1:172.21.15.178:6800/4205070280 conn(0x7f977006ec00 legacy=0x7f97700710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:30.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.674+0000 7f979007a700 1 -- 172.21.15.178:0/1447999510 >> v1:172.21.15.178:6789/0 conn(0x7f97881097b0 legacy=0x7f97881a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:30.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.674+0000 7f979007a700 1 -- 172.21.15.178:0/1447999510 shutdown_connections 2024-02-14T17:43:30.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.674+0000 7f979007a700 1 -- 172.21.15.178:0/1447999510 >> 172.21.15.178:0/1447999510 conn(0x7f97880fc410 msgr2=0x7f97881108a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:30.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.674+0000 7f979007a700 1 -- 172.21.15.178:0/1447999510 shutdown_connections 2024-02-14T17:43:30.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-02-14T17:43:30.674+0000 7f979007a700 1 -- 172.21.15.178:0/1447999510 wait complete. 2024-02-14T17:43:30.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:30 smithi184 ceph-mon[33536]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:30.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:30 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:31.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:43:30 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:43:31.363 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-14T17:43:31.363 INFO:tasks.cephadm:Setup complete, yielding 2024-02-14T17:43:31.364 INFO:teuthology.run_tasks:Running task workunit... 2024-02-14T17:43:31.377 INFO:tasks.workunit:Pulling workunits from ref 8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:43:31.378 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-14T17:43:31.378 DEBUG:teuthology.orchestra.run.smithi178:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-14T17:43:31.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:43:31.395 INFO:teuthology.orchestra.run.smithi178.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-14T17:43:31.396 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-14T17:43:31.452 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-14T17:43:31.452 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-14T17:43:31.508 INFO:tasks.workunit:timeout=3h 2024-02-14T17:43:31.508 INFO:tasks.workunit:cleanup=True 2024-02-14T17:43:31.508 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:43:31.564 INFO:tasks.workunit.client.0.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-14T17:43:31.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1447999510' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T17:43:31.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:31 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1447999510' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T17:43:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:31 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:31.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1447999510' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T17:43:31.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:31 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:32.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:32 smithi184 ceph-mon[33536]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:32.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:32 smithi178 ceph-mon[39811]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:32 smithi178 ceph-mon[30321]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:34 smithi184 ceph-mon[33536]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:34 smithi178 ceph-mon[39811]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:34.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:34 smithi178 ceph-mon[30321]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:35.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:43:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:43:35.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:43:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78060]: ts=2024-02-14T17:43:35.531Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002378017s 2024-02-14T17:43:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:36 smithi184 ceph-mon[33536]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:36.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:36 smithi178 ceph-mon[39811]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:36 smithi178 ceph-mon[30321]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:38.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:38 smithi184 ceph-mon[33536]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:38.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:38 smithi178 ceph-mon[39811]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:38 smithi178 ceph-mon[30321]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:40.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:40 smithi184 ceph-mon[33536]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:40.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:40 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:40 smithi178 ceph-mon[39811]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:40 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:40 smithi178 ceph-mon[30321]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:40 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:41.143 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:43:40 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:43:41.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:41 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:41 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:41 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:42 smithi184 ceph-mon[33536]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:42.720 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:42 smithi184 systemd[1]: Starting Ceph grafana.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:43:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:42 smithi178 ceph-mon[39811]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:42 smithi178 ceph-mon[30321]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:43.023 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:42 smithi184 podman[66707]: 2024-02-14 17:43:42.720729993 +0000 UTC m=+0.111454230 container create 03ec241e3c5ec5e2054df2e49810da0ca652b1f62b6859c11a2549ea32b3bd70 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a, io.openshift.expose-services=, release=239.1651231664, io.buildah.version=1.32.0, io.openshift.tags=base rhel8, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git, build-date=2022-04-29T11:48:11.055897, maintainer=Paul Cuzner , com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, description=Ceph Grafana Container, distribution-scope=public, name=ubi8) 2024-02-14T17:43:43.024 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:42 smithi184 podman[66707]: 2024-02-14 17:43:42.626048926 +0000 UTC m=+0.016773182 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-02-14T17:43:43.024 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:42 smithi184 podman[66707]: 2024-02-14 17:43:42.922882442 +0000 UTC m=+0.313606697 container init 03ec241e3c5ec5e2054df2e49810da0ca652b1f62b6859c11a2549ea32b3bd70 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, maintainer=Paul Cuzner , build-date=2022-04-29T11:48:11.055897, distribution-scope=public, vendor=Red Hat, Inc., io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, name=ubi8, release=239.1651231664, architecture=x86_64, version=8.5, io.openshift.expose-services=, com.redhat.component=ubi8-container, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8) 2024-02-14T17:43:43.024 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:42 smithi184 podman[66707]: 2024-02-14 17:43:42.928256162 +0000 UTC m=+0.318980401 container start 03ec241e3c5ec5e2054df2e49810da0ca652b1f62b6859c11a2549ea32b3bd70 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, description=Ceph Grafana Container, vcs-type=git, distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , release=239.1651231664, io.openshift.expose-services=, version=8.5, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897) 2024-02-14T17:43:43.276 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 bash[66707]: 03ec241e3c5ec5e2054df2e49810da0ca652b1f62b6859c11a2549ea32b3bd70 2024-02-14T17:43:43.276 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 systemd[1]: Started Ceph grafana.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:43:43.276 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.0899008Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-14T17:43:43.277 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090131672Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-14T17:43:43.277 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090157418Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-14T17:43:43.277 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090169125Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-14T17:43:43.277 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090179984Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-14T17:43:43.277 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090188006Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-14T17:43:43.277 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090193906Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-14T17:43:43.277 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090199694Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-14T17:43:43.277 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090205435Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-14T17:43:43.277 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090211425Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-14T17:43:43.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090219716Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-14T17:43:43.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090229755Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-14T17:43:43.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090239506Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-14T17:43:43.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=settings t=2024-02-14T17:43:43.090249573Z level=info msg="App mode production" 2024-02-14T17:43:43.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=sqlstore t=2024-02-14T17:43:43.090311982Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-14T17:43:43.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=sqlstore t=2024-02-14T17:43:43.090336355Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-14T17:43:43.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.09139849Z level=info msg="Starting DB migrations" 2024-02-14T17:43:43.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.09150736Z level=info msg="Executing migration" id="create migration_log table" 2024-02-14T17:43:43.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.092173094Z level=info msg="Executing migration" id="create user table" 2024-02-14T17:43:43.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.092689182Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-14T17:43:43.278 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.093204689Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.093760804Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.094373735Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.094788814Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.095508081Z level=info msg="Executing migration" id="create user table v2" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.0959476Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.096335805Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.096793377Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.097133427Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.097451121Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.097870521Z level=info msg="Executing migration" id="Update user table charset" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.098135259Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-14T17:43:43.279 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.098518291Z level=info msg="Executing migration" id="Add missing user data" 2024-02-14T17:43:43.280 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.098866903Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-14T17:43:43.280 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.099257108Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-14T17:43:43.280 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.099650896Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-14T17:43:43.280 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.100058878Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-14T17:43:43.281 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.101186706Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-14T17:43:43.281 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.101624614Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-14T17:43:43.281 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.102074369Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-14T17:43:43.281 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.102464647Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-14T17:43:43.281 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.102855907Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-14T17:43:43.281 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.103228638Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-14T17:43:43.281 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.103486165Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-14T17:43:43.281 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.103892913Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.10427057Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.104679542Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.10510163Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.107601781Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.10815399Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.108547027Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.108984557Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.109360697Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.109778111Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.110153032Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.110490936Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.110862848Z level=info msg="Executing migration" id="create star table" 2024-02-14T17:43:43.282 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.111220903Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.111651128Z level=info msg="Executing migration" id="create org table v1" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.112084243Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.112493879Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.112925077Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.113335763Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.113764607Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.114191461Z level=info msg="Executing migration" id="Update org table charset" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.114468553Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.114808039Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.115118709Z level=info msg="Executing migration" id="create dashboard table" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.115514162Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.115941501Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.11637131Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.116802375Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.117253985Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-14T17:43:43.283 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.119970659Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.122633256Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.125421076Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.128046643Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.130824581Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.133458197Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.133968407Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.134263452Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.134785835Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.135339651Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.136186197Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.137063261Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.137865109Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.138332474Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-14T17:43:43.284 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.141208427Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.143854148Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.146695269Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.14728587Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.150259921Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.153125117Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.15584183Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.156208993Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.159019546Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.161511497Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.164216218Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.164697559Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.167465244Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.170180244Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.172759927Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-14T17:43:43.285 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.173317299Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-14T17:43:43.286 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.176029638Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-14T17:43:43.286 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.178800155Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-14T17:43:43.286 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.18147736Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-14T17:43:43.286 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.182134424Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-14T17:43:43.286 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.184468522Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-14T17:43:43.286 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.184849514Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-14T17:43:43.286 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.187035074Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-14T17:43:43.286 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.187643758Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-14T17:43:43.286 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.188186099Z level=info msg="Executing migration" id="create data_source table" 2024-02-14T17:43:43.286 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.188641973Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-14T17:43:43.286 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.189418657Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.190380589Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.191116672Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.191528023Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.192571729Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.19303904Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.19383018Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.19431129Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.194702243Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.195293813Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.196115485Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.196895617Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-14T17:43:43.287 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.197727799Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.198615372Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.199570336Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.200294144Z level=info msg="Executing migration" id="Add uid column" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.201290819Z level=info msg="Executing migration" id="Update uid value" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.201860984Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.202718788Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.203722605Z level=info msg="Executing migration" id="create api_key table" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.204397488Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.20487717Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.205787872Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.2065553Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.207538224Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.208357398Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-14T17:43:43.288 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.209092461Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.210201297Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.211199622Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.211960503Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.212909805Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.215763008Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.218582823Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.221581419Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.224125676Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.226907355Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.229795325Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.232552793Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.235403986Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.238313688Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.241069472Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-14T17:43:43.289 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.243773481Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.246603947Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.249307514Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.252205267Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.254983014Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.257751356Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.260489938Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.263582255Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.266245494Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.268944703Z level=info msg="Executing migration" id="create quota table v1" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.271474344Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.272055186Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.274869344Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.275488623Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-14T17:43:43.290 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.277514152Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-14T17:43:43.527 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.280203221Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-14T17:43:43.527 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.280541344Z level=info msg="Executing migration" id="create session table" 2024-02-14T17:43:43.527 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.283273741Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-14T17:43:43.527 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.28593684Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-14T17:43:43.527 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.28834085Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.288772112Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.291662676Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.294359016Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.29711246Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.297590686Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.298465538Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.299308487Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.300182683Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.300937744Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.301778449Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.302596943Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-14T17:43:43.528 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.303432445Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.303771596Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.304677712Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.305510859Z level=info msg="Executing migration" id="create alert table v1" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.306307847Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.306780311Z level=info msg="Executing migration" id="add index alert state" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.307745909Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.308402408Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.309244505Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.30967185Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.310526994Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.31192357Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.312876368Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.313570225Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-14T17:43:43.529 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.314516776Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-14T17:43:43.530 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.315259151Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-14T17:43:43.530 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.316078691Z level=info msg="Executing migration" id="Add column is_default" 2024-02-14T17:43:43.530 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.316711963Z level=info msg="Executing migration" id="Add column frequency" 2024-02-14T17:43:43.530 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.317325344Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-14T17:43:43.530 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.31817377Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-14T17:43:43.530 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.319039378Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-14T17:43:43.530 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.319910438Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-14T17:43:43.531 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.320211561Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.321078033Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.321484052Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.3219461Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.322868556Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.323457408Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.323875774Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.326692205Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.327501468Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.329573025Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.332267539Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.33277157Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-14T17:43:43.532 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.335638847Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.338321156Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.341075753Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.341675741Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.341980852Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.342444032Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.342932922Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.343373538Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.343844669Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.344360877Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.345351787Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.34579633Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.346414634Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-14T17:43:43.533 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.346836546Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.347483801Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.348396323Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.348883144Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.349858172Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.351641274Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.354577032Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.355044197Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.357930348Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.358600564Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.360847634Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.363762494Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.366608204Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-14T17:43:43.534 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.369500557Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.372484453Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.375259488Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.375990997Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.378920633Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.381766341Z level=info msg="Executing migration" id="Move region to single row" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.384479445Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.385027129Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.387972849Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.388446553Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.390649019Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.393547858Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-14T17:43:43.535 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.394111362Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.39661252Z level=info msg="Executing migration" id="create test_data table" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.397032887Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.399429766Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.402159236Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.404810978Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.407444772Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.41009014Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.412665697Z level=info msg="Executing migration" id="create team table" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.413062304Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.415745933Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.418315691Z level=info msg="Executing migration" id="create team member table" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.421114659Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.421707116Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.42462012Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-14T17:43:43.536 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.427390983Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.430047582Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.430787769Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.431612351Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.43246174Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.433331666Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.433763962Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.43418924Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.435186731Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.435867954Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.436693599Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.437695981Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.438409825Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.439273647Z level=info msg="Executing migration" id="create tag table" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.440097488Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.441025019Z level=info msg="Executing migration" id="create login attempt table" 2024-02-14T17:43:43.537 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.441821969Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.442640573Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.443598162Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.44548659Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.44647995Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.447220178Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.448041342Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.448437803Z level=info msg="Executing migration" id="create user auth table" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.44895521Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.449803365Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.450787766Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.451785312Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.452607719Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-14T17:43:43.538 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.453491832Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.454388114Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.455366599Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.456193519Z level=info msg="Executing migration" id="create server_lock table" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.457080272Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.458045637Z level=info msg="Executing migration" id="create user auth token table" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.458799998Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.459755663Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.460538626Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.463352898Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.466109745Z level=info msg="Executing migration" id="create cache_data table" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.469094101Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.471848731Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.474644552Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.477565761Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.480267588Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-14T17:43:43.539 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.480611996Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.481058713Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.483765375Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.48421018Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.484476089Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.484893586Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.485262591Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.485673432Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.486091807Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.486889681Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.487315253Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.487587154Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.487987157Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.488387308Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.4888133Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.489092078Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-14T17:43:43.540 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.48949249Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.489902661Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.490288582Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.490723711Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.49148917Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.491903111Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.492285073Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.495577621Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.498894918Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.499318574Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.499730849Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.500740011Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.50118761Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.501606843Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.502020815Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.502420777Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.502721729Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-14T17:43:43.541 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.503522054Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.504311491Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.505114696Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.505546546Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.505997518Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.50680696Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.507630228Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.508072293Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.508931112Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.509776584Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.510118039Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.510568439Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.513415529Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.516399076Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.519085725Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.519982165Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.522851567Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-14T17:43:43.542 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.525622346Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-14T17:43:43.543 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.52822841Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-14T17:43:43.543 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.529490609Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-14T17:43:43.808 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.53222137Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-14T17:43:43.808 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.534707087Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.537403069Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.537707269Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.540421725Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.543234692Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.545981592Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.546408236Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.549268591Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.551857292Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.554601008Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.555065045Z level=info msg="Executing migration" id="create alert_image table" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.557619706Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.560385037Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-14T17:43:43.809 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.56298545Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-14T17:43:43.810 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.563398527Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-14T17:43:43.810 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.566143971Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-14T17:43:43.810 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.566237912Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-14T17:43:43.810 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.568964199Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.571451883Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.57193221Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.572065812Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.57217552Z level=info msg="alerts found to migrate" alerts=0 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.572757429Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.573222361Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.573657244Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.574556763Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.575203403Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.576044805Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.576889859Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-14T17:43:43.811 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.577824376Z level=info msg="Executing migration" id="create data_keys table" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.578444848Z level=info msg="Executing migration" id="create secrets table" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.579279732Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.583779016Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.5847243Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.585034053Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.589217862Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.593360725Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.594195163Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.594652744Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.595659147Z level=info msg="Executing migration" id="create permission table" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.596352977Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-14T17:43:43.812 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.597203356Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.597686032Z level=info msg="Executing migration" id="create role table" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.598567805Z level=info msg="Executing migration" id="add column display_name" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.599481689Z level=info msg="Executing migration" id="add column group_name" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.600400633Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.60086425Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.601673849Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.602685087Z level=info msg="Executing migration" id="create team role table" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.60338849Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.60383163Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.604650807Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.605574966Z level=info msg="Executing migration" id="create user role table" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.606422797Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.606898691Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-14T17:43:43.813 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.607768284Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.610663374Z level=info msg="Executing migration" id="create builtin role table" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.613573096Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.614097997Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.616803019Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.61976574Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.622550765Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.62306258Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.625792228Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.626408025Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.628571233Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.631358235Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.632363615Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-14T17:43:43.814 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.635062081Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-14T17:43:43.815 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.637928128Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-14T17:43:43.815 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.640703943Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-14T17:43:43.815 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.641236946Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-14T17:43:43.815 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.644162619Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-14T17:43:43.815 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.646970993Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-14T17:43:43.816 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.649803511Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-14T17:43:43.816 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.65023433Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-14T17:43:43.816 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.652962414Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-14T17:43:43.816 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.655760902Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-14T17:43:43.816 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.658471496Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-14T17:43:43.816 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.658842998Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-14T17:43:43.816 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.661619024Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-14T17:43:43.816 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.664307032Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-14T17:43:43.816 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.667151446Z level=info msg="Executing migration" id="add correlation config column" 2024-02-14T17:43:43.816 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.668255296Z level=info msg="Executing migration" id="create entity_events table" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.670942062Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.673367606Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.673509583Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.675927275Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.676023371Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.67630721Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.679186088Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.681778598Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.684679653Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.685544074Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.686241266Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.687082781Z level=info msg="Executing migration" id="Drop public config table" 2024-02-14T17:43:43.817 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.687971369Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.688818233Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.689681477Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.6905579Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.691370359Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.694278579Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.695287827Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.696291356Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.697166364Z level=info msg="Executing migration" id="add share column" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.69816418Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.699023382Z level=info msg="Executing migration" id="create file table" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.70006998Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.700888844Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.701620043Z level=info msg="Executing migration" id="create file_meta table" 2024-02-14T17:43:43.818 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.702492539Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.703528048Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.704359732Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.705115994Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.70594587Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.706386701Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.707418318Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.708234401Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.709074857Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.70995696Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.71029879Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.711282704Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.712537093Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.713568179Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-14T17:43:43.819 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.714016392Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.723589204Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.726458327Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.729371401Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.732874046Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.735854096Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.738670582Z level=info msg="Executing migration" id="create folder table" 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.741161798Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.741614702Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.744480538Z level=info msg="Executing migration" id="Update folder title length" 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.747196809Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=migrator t=2024-02-14T17:43:43.749869594Z level=info msg="migrations completed" performed=485 skipped=0 duration=658.398537ms 2024-02-14T17:43:43.820 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=sqlstore t=2024-02-14T17:43:43.750461939Z level=info msg="Created default organization" 2024-02-14T17:43:43.821 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=plugin.loader t=2024-02-14T17:43:43.807179067Z level=info msg="Plugin registered" pluginID=input 2024-02-14T17:43:44.086 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=plugin.loader t=2024-02-14T17:43:43.83493988Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-14T17:43:44.086 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=plugin.loader t=2024-02-14T17:43:43.834965079Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-14T17:43:44.086 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=secrets t=2024-02-14T17:43:43.835009858Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-14T17:43:44.086 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=query_data t=2024-02-14T17:43:43.83708833Z level=info msg="Query Service initialization" 2024-02-14T17:43:44.086 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=live.push_http t=2024-02-14T17:43:43.839421021Z level=info msg="Live Push Gateway initialization" 2024-02-14T17:43:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:44 smithi178 ceph-mon[30321]: Deploying daemon node-exporter.a on smithi178 2024-02-14T17:43:44.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:44 smithi178 ceph-mon[39811]: Deploying daemon node-exporter.a on smithi178 2024-02-14T17:43:44.334 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:44 smithi178 bash[78785]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-14T17:43:44.444 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:44.445 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-mon[33536]: Deploying daemon node-exporter.a on smithi178 2024-02-14T17:43:44.720 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=infra.usagestats.collector t=2024-02-14T17:43:44.443882194Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-14T17:43:44.720 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=provisioning.datasources t=2024-02-14T17:43:44.444948865Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-14T17:43:44.720 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=provisioning.datasources t=2024-02-14T17:43:44.456362031Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-14T17:43:44.720 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=provisioning.alerting t=2024-02-14T17:43:44.467114217Z level=info msg="starting to provision alerting" 2024-02-14T17:43:44.720 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=provisioning.alerting t=2024-02-14T17:43:44.467133178Z level=info msg="finished to provision alerting" 2024-02-14T17:43:44.720 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=grafanaStorageLogger t=2024-02-14T17:43:44.467287219Z level=info msg="storage starting" 2024-02-14T17:43:44.720 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=ngalert.state.manager t=2024-02-14T17:43:44.467477764Z level=info msg="Warming state cache for startup" 2024-02-14T17:43:44.720 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=http.server t=2024-02-14T17:43:44.469066511Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-14T17:43:44.720 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=ngalert.state.manager t=2024-02-14T17:43:44.478358621Z level=info msg="State cache has been initialized" states=0 duration=10.88016ms 2024-02-14T17:43:44.721 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=ticker t=2024-02-14T17:43:44.478416062Z level=info msg=starting first_tick=2024-02-14T17:43:50Z 2024-02-14T17:43:44.721 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=ngalert.multiorg.alertmanager t=2024-02-14T17:43:44.47841125Z level=info msg="starting MultiOrg Alertmanager" 2024-02-14T17:43:44.721 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:43:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=sqlstore.transactions t=2024-02-14T17:43:44.483194219Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-14T17:43:45.082 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:44 smithi178 bash[78785]: Getting image source signatures 2024-02-14T17:43:45.083 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:44 smithi178 bash[78785]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-14T17:43:45.083 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:44 smithi178 bash[78785]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-14T17:43:45.083 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:44 smithi178 bash[78785]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-14T17:43:45.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:45 smithi184 ceph-mon[33536]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:45.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:45 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:45.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:45 smithi178 ceph-mon[39811]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:45.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:45 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:45.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:45 smithi178 ceph-mon[30321]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:45.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:45 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:46.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:43:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:43:46.348 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 bash[78785]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-14T17:43:46.694 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 bash[78785]: Writing manifest to image destination 2024-02-14T17:43:46.961 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 podman[78785]: 2024-02-14 17:43:46.603089978 +0000 UTC m=+2.432725061 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-14T17:43:46.961 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 podman[78785]: 2024-02-14 17:43:46.758101941 +0000 UTC m=+2.587737016 container create 591f4445a1bdc52086d54c89c1f062e0b3b2340419c0e71c50034a21cff930f6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T17:43:47.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:47 smithi178 ceph-mon[39811]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:47.293 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:47 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:47.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:47 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:47.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:47 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:47.294 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:47 smithi178 ceph-mon[30321]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:47.294 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:47 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:47.294 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:47 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:47.294 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:47 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:47.295 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 podman[78785]: 2024-02-14 17:43:46.960035097 +0000 UTC m=+2.789670180 container init 591f4445a1bdc52086d54c89c1f062e0b3b2340419c0e71c50034a21cff930f6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T17:43:47.295 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 podman[78785]: 2024-02-14 17:43:46.965664958 +0000 UTC m=+2.795300034 container start 591f4445a1bdc52086d54c89c1f062e0b3b2340419c0e71c50034a21cff930f6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T17:43:47.295 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.969Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-14T17:43:47.295 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.969Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-14T17:43:47.295 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.969Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-14T17:43:47.295 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.969Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-14T17:43:47.296 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-14T17:43:47.296 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-14T17:43:47.296 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-14T17:43:47.296 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=arp 2024-02-14T17:43:47.296 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-14T17:43:47.296 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-14T17:43:47.296 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-14T17:43:47.296 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-14T17:43:47.296 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-14T17:43:47.296 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-14T17:43:47.296 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=edac 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-14T17:43:47.297 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=os 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=stat 2024-02-14T17:43:47.298 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=time 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=uname 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:46 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a[78846]: ts=2024-02-14T17:43:46.970Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:47 smithi178 bash[78785]: 591f4445a1bdc52086d54c89c1f062e0b3b2340419c0e71c50034a21cff930f6 2024-02-14T17:43:47.299 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 17:43:47 smithi178 systemd[1]: Started Ceph node-exporter.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:43:47.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:47 smithi184 ceph-mon[33536]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:47.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:47 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:47.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:47 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:47.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:47 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:48.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:48 smithi184 ceph-mon[33536]: Deploying daemon node-exporter.b on smithi184 2024-02-14T17:43:48.470 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:48 smithi184 bash[66996]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-14T17:43:48.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:48 smithi178 ceph-mon[39811]: Deploying daemon node-exporter.b on smithi184 2024-02-14T17:43:48.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:48 smithi178 ceph-mon[30321]: Deploying daemon node-exporter.b on smithi184 2024-02-14T17:43:49.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:49 smithi184 ceph-mon[33536]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:49.470 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:49 smithi184 bash[66996]: Getting image source signatures 2024-02-14T17:43:49.470 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:49 smithi184 bash[66996]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-14T17:43:49.470 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:49 smithi184 bash[66996]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-14T17:43:49.470 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:49 smithi184 bash[66996]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-14T17:43:49.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:49 smithi178 ceph-mon[39811]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:49.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:49 smithi178 ceph-mon[30321]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:51.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:43:50 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:43:51.220 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 bash[66996]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-14T17:43:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-mon[33536]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:51.499 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 bash[66996]: Writing manifest to image destination 2024-02-14T17:43:51.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:51 smithi178 ceph-mon[30321]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:51.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:51 smithi178 ceph-mon[39811]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:51.833 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 podman[66996]: 2024-02-14 17:43:51.481652355 +0000 UTC m=+3.314398250 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-14T17:43:51.833 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 podman[66996]: 2024-02-14 17:43:51.58023388 +0000 UTC m=+3.412979773 container create daa702fa9f4132991e4628270d71c53c2fec07851d2b5ae315af8aa95fa940e2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T17:43:52.146 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 podman[66996]: 2024-02-14 17:43:51.832534023 +0000 UTC m=+3.665279920 container init daa702fa9f4132991e4628270d71c53c2fec07851d2b5ae315af8aa95fa940e2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T17:43:52.146 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 podman[66996]: 2024-02-14 17:43:51.83861778 +0000 UTC m=+3.671363674 container start daa702fa9f4132991e4628270d71c53c2fec07851d2b5ae315af8aa95fa940e2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T17:43:52.146 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.842Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-14T17:43:52.146 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.842Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-14T17:43:52.146 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-14T17:43:52.146 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-14T17:43:52.146 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-14T17:43:52.146 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-14T17:43:52.147 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-14T17:43:52.147 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=arp 2024-02-14T17:43:52.147 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-14T17:43:52.147 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-14T17:43:52.147 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-14T17:43:52.147 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-14T17:43:52.147 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-14T17:43:52.148 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-14T17:43:52.148 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-14T17:43:52.148 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-14T17:43:52.148 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=edac 2024-02-14T17:43:52.148 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-14T17:43:52.148 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-14T17:43:52.149 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=os 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=stat 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=time 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-14T17:43:52.150 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=uname 2024-02-14T17:43:52.151 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-14T17:43:52.151 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-14T17:43:52.151 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.843Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-14T17:43:52.151 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.844Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-14T17:43:52.151 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b[67046]: ts=2024-02-14T17:43:51.844Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-14T17:43:52.151 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 bash[66996]: daa702fa9f4132991e4628270d71c53c2fec07851d2b5ae315af8aa95fa940e2 2024-02-14T17:43:52.151 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 17:43:51 smithi184 systemd[1]: Started Ceph node-exporter.b for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:43:52.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:52 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:52.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:52 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:52 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:52 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:52 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:52 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:52.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:43:52.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:52.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:52 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:53 smithi184 ceph-mon[33536]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:53 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:53 smithi178 ceph-mon[30321]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:53 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:53.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:53 smithi178 ceph-mon[39811]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:53.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:53 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:55.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:55 smithi178 ceph-mon[39811]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:55.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:55 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:55.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:55 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:55.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:55 smithi178 ceph-mon[30321]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:55.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:55 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:55.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:55 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:55.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:55 smithi184 ceph-mon[33536]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:55.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:55 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:55.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:55 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:43:56.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:43:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:43:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:43:56.768 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:56 smithi178 ceph-mon[39811]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:56.768 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:56 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:56.768 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:56 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:56.769 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:56 smithi178 ceph-mon[30321]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:56.769 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:56 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:56.769 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:56 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:56 smithi184 ceph-mon[33536]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:43:56.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:56 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:56.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:56 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:43:58.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:43:58 smithi178 ceph-mon[39811]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:43:58 smithi178 ceph-mon[30321]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:43:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:43:58 smithi184 ceph-mon[33536]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:00 smithi184 ceph-mon[33536]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:00 smithi178 ceph-mon[39811]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:00 smithi178 ceph-mon[30321]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:01.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:44:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:01 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:01 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:01.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:01 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:02.795 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[39811]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:02.796 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:02.796 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:02.796 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:02.796 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:02.796 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:02.796 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[39811]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-14T17:44:02.796 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[39811]: Reconfiguring daemon alertmanager.a on smithi178 2024-02-14T17:44:02.796 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[30321]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:02.796 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:02.797 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:02.797 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:02.797 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:02.797 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:02.797 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[30321]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-14T17:44:02.797 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:02 smithi178 ceph-mon[30321]: Reconfiguring daemon alertmanager.a on smithi178 2024-02-14T17:44:02.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:02 smithi184 ceph-mon[33536]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:02 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:02 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:02 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:02 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:02 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:02 smithi184 ceph-mon[33536]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-14T17:44:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:02 smithi184 ceph-mon[33536]: Reconfiguring daemon alertmanager.a on smithi178 2024-02-14T17:44:04.783 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:04 smithi178 ceph-mon[30321]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:04.783 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:04 smithi178 ceph-mon[39811]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:04.790 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:04 smithi184 ceph-mon[33536]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:05.959 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:44:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:44:05.960 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:05 smithi178 systemd[1]: Stopping Ceph alertmanager.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:44:06.525 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:06 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78060]: ts=2024-02-14T17:44:06.241Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-14T17:44:06.525 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:06 smithi178 podman[79662]: 2024-02-14 17:44:06.261692909 +0000 UTC m=+0.148216452 container died 8c8a8c324cce86cf34c6a36cef46b7d3b40bd9bc78f5b69b3ac4ed5b800967d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:06.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:06 smithi178 ceph-mon[39811]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:06 smithi178 ceph-mon[30321]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:06.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:06 smithi178 podman[79662]: 2024-02-14 17:44:06.663818245 +0000 UTC m=+0.550341788 container cleanup 8c8a8c324cce86cf34c6a36cef46b7d3b40bd9bc78f5b69b3ac4ed5b800967d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:06.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:06 smithi178 bash[79662]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a 2024-02-14T17:44:06.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:06 smithi184 ceph-mon[33536]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:07.243 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:06 smithi178 podman[79681]: 2024-02-14 17:44:06.957210234 +0000 UTC m=+0.698773383 container remove 8c8a8c324cce86cf34c6a36cef46b7d3b40bd9bc78f5b69b3ac4ed5b800967d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:07.243 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 podman[79681]: 2024-02-14 17:44:07.057933704 +0000 UTC m=+0.799496855 volume remove 9b0d8921dfb3160ac6fa266a23fc1c4876bec99703bfdebbc673f64459e714cd 2024-02-14T17:44:07.582 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-14T17:44:07.583 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 systemd[1]: Stopped Ceph alertmanager.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:44:07.583 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 systemd[1]: Starting Ceph alertmanager.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:44:07.583 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 podman[79783]: 2024-02-14 17:44:07.376592773 +0000 UTC m=+0.018656071 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-02-14T17:44:07.583 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 podman[79783]: 2024-02-14 17:44:07.50052753 +0000 UTC m=+0.142590816 volume create 7877abb90a56773027160f5a13f00a76de62feee3dbfe74676017e64054f9dd7 2024-02-14T17:44:07.971 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 podman[79783]: 2024-02-14 17:44:07.609404773 +0000 UTC m=+0.251468058 container create 68f4ed8d5cca2c4a42ccaab3df368d4637d4fb97653b1892651a11cf197b9db3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:08.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 podman[79783]: 2024-02-14 17:44:07.971066129 +0000 UTC m=+0.613129426 container init 68f4ed8d5cca2c4a42ccaab3df368d4637d4fb97653b1892651a11cf197b9db3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:08.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 podman[79783]: 2024-02-14 17:44:07.978640323 +0000 UTC m=+0.620703608 container start 68f4ed8d5cca2c4a42ccaab3df368d4637d4fb97653b1892651a11cf197b9db3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:08.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T17:44:07.990Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-14T17:44:08.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T17:44:07.990Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-14T17:44:08.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T17:44:07.991Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.178 port=9094 2024-02-14T17:44:08.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:07 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T17:44:07.992Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-14T17:44:08.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:08 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T17:44:08.015Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-14T17:44:08.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:08 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T17:44:08.015Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-14T17:44:08.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:08 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T17:44:08.018Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-14T17:44:08.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:08 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T17:44:08.018Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-14T17:44:08.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:08 smithi178 bash[79783]: 68f4ed8d5cca2c4a42ccaab3df368d4637d4fb97653b1892651a11cf197b9db3 2024-02-14T17:44:08.334 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:08 smithi178 systemd[1]: Started Ceph alertmanager.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:44:09.179 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:09 smithi184 ceph-mon[33536]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:09.180 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:09 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:09.180 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:09 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:09.180 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:09 smithi184 ceph-mon[33536]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-14T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:09 smithi178 ceph-mon[30321]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:09 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:09 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:09 smithi178 ceph-mon[30321]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-14T17:44:09.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:09 smithi178 ceph-mon[39811]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:09.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:09 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:09.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:09 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:09.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:09 smithi178 ceph-mon[39811]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-14T17:44:10.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:10 smithi178 ceph-mon[39811]: Reconfiguring daemon prometheus.a on smithi184 2024-02-14T17:44:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:10 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:10.333 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:09 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T17:44:09.992Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000749312s 2024-02-14T17:44:10.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:10 smithi178 ceph-mon[30321]: Reconfiguring daemon prometheus.a on smithi184 2024-02-14T17:44:10.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:10 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:10.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:10 smithi184 ceph-mon[33536]: Reconfiguring daemon prometheus.a on smithi184 2024-02-14T17:44:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:10 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:11.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-mon[33536]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:11.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:44:10 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:44:11.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:11 smithi178 ceph-mon[39811]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:11.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:11 smithi178 ceph-mon[30321]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:11.677 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 systemd[1]: Stopping Ceph prometheus.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:44:11.950 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.675Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-14T17:44:11.950 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.675Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-14T17:44:11.950 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.675Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-14T17:44:11.950 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.675Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-14T17:44:11.950 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.675Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-14T17:44:11.950 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.675Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-14T17:44:11.950 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.675Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-14T17:44:11.950 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.675Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-14T17:44:11.950 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.675Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-14T17:44:11.950 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.676Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-14T17:44:11.951 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.676Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-14T17:44:11.951 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[64712]: ts=2024-02-14T17:44:11.676Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-14T17:44:11.951 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:11 smithi184 podman[68089]: 2024-02-14 17:44:11.686593661 +0000 UTC m=+0.122550166 container died 52997012da916155824f86293051dbb0eeab547f947af108b68f5528d309d39b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:12.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:12.220 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 podman[68089]: 2024-02-14 17:44:12.038985519 +0000 UTC m=+0.474942024 container cleanup 52997012da916155824f86293051dbb0eeab547f947af108b68f5528d309d39b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:12.220 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 bash[68089]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a 2024-02-14T17:44:12.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:12 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:12.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:12 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:12.662 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 podman[68107]: 2024-02-14 17:44:12.315257287 +0000 UTC m=+0.631534419 container remove 52997012da916155824f86293051dbb0eeab547f947af108b68f5528d309d39b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:12.662 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-02-14T17:44:12.662 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 systemd[1]: Stopped Ceph prometheus.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:44:12.662 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@prometheus.a.service: Consumed 1.186s CPU time. 2024-02-14T17:44:12.662 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 systemd[1]: Starting Ceph prometheus.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:44:12.932 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 podman[68213]: 2024-02-14 17:44:12.564623794 +0000 UTC m=+0.019446581 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-14T17:44:12.932 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 podman[68213]: 2024-02-14 17:44:12.674663733 +0000 UTC m=+0.129486515 container create a00d25be6bcb9a0e778ef83ead3d5f3d552c30d10a678543370a58f4037707dc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:12.932 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 podman[68213]: 2024-02-14 17:44:12.901825193 +0000 UTC m=+0.356647976 container init a00d25be6bcb9a0e778ef83ead3d5f3d552c30d10a678543370a58f4037707dc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:12.932 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 podman[68213]: 2024-02-14 17:44:12.907205744 +0000 UTC m=+0.362028532 container start a00d25be6bcb9a0e778ef83ead3d5f3d552c30d10a678543370a58f4037707dc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:44:13.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:13 smithi184 ceph-mon[33536]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:13 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:13 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:44:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi178.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:44:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:13 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:44:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:44:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:13 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:44:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:44:13.221 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.931Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-14T17:44:13.221 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.931Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-14T17:44:13.221 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.931Z caller=main.go:570 level=info host_details="(Linux 5.14.0-420.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 9 18:10:28 UTC 2024 x86_64 smithi184 (none))" 2024-02-14T17:44:13.221 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.931Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-14T17:44:13.221 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.931Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.933Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.933Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.934Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.934Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.936Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.936Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=3.598µs 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.936Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.937Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.937Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.937Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=41.647µs wal_replay_duration=1.297922ms wbl_replay_duration=101ns total_replay_duration=1.417485ms 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.938Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.938Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.938Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.956Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=18.45395ms db_storage=1.013µs remote_storage=1.028µs web_handler=358ns query_engine=503ns scrape=461.923µs scrape_sd=88.996µs notify=21.13µs notify_sd=22.525µs rules=17.65179ms tracing=5.373µs 2024-02-14T17:44:13.222 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.956Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-14T17:44:13.223 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T17:44:12.956Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-14T17:44:13.223 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:13 smithi184 bash[68213]: a00d25be6bcb9a0e778ef83ead3d5f3d552c30d10a678543370a58f4037707dc 2024-02-14T17:44:13.223 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 17:44:13 smithi184 systemd[1]: Started Ceph prometheus.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:44:13.446 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STOPPING 2024-02-14T17:44:13.446 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T17:44:13.446 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STOPPED 2024-02-14T17:44:13.446 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STARTING 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Serving on http://:::9283 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STARTED 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STOPPING 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STOPPED 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STARTING 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Serving on http://:::9283 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STARTED 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STOPPING 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STOPPED 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STARTING 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Serving on http://:::9283 2024-02-14T17:44:13.447 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:17:44:13] ENGINE Bus STARTED 2024-02-14T17:44:13.448 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[39811]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:13.448 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.448 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.448 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:44:13.448 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi178.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:44:13.448 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.448 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:44:13.448 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:44:13.448 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.448 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:44:13.448 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:44:13.449 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[30321]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:13.449 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.449 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.449 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:44:13.449 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi178.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:44:13.449 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.449 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:44:13.449 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:44:13.449 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:13.449 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:44:13.449 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:44:14.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:14 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:44:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:14 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi178.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:44:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:14 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:44:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:14 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:44:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:14 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:44:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:14 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:44:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:14 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:14 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:14.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:44:14.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi178.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:44:14.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:44:14.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:44:14.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:44:14.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:44:14.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:14.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:14.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:44:14.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi178.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:44:14.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:44:14.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:44:14.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:44:14.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:44:14.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:14.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:14 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:15.169 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:15 smithi178 ceph-mon[39811]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:15.169 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:15 smithi178 ceph-mon[30321]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:15 smithi184 ceph-mon[33536]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:17.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:17 smithi178 ceph-mon[30321]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:17.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:17 smithi178 ceph-mon[39811]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:17.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:17 smithi184 ceph-mon[33536]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:18.332 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 17:44:17 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T17:44:17.995Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.00379042s 2024-02-14T17:44:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[39811]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[30321]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:18 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:18 smithi184 ceph-mon[33536]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:18 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:18 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:18 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:18 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:18 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:18 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:18 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:20.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:20 smithi178 ceph-mon[39811]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:20 smithi178 ceph-mon[30321]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:20.926 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:20 smithi184 ceph-mon[33536]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:21.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:44:20 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:44:21.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:21 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:21.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:21 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:21.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:21 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:22.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:22 smithi178 ceph-mon[39811]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:22 smithi178 ceph-mon[30321]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:22 smithi184 ceph-mon[33536]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:25 smithi184 ceph-mon[33536]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:25.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:25 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:25.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:25 smithi178 ceph-mon[39811]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:25.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:25 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:25 smithi178 ceph-mon[30321]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:25 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:26.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:44:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:44:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:27 smithi184 ceph-mon[33536]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:27.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:27 smithi178 ceph-mon[39811]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:27.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:27 smithi178 ceph-mon[30321]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:29 smithi184 ceph-mon[33536]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:29.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:29 smithi178 ceph-mon[39811]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:29.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:29 smithi178 ceph-mon[30321]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:31.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:31 smithi184 ceph-mon[33536]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:31.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:44:30 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:44:31.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:31 smithi178 ceph-mon[39811]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:31 smithi178 ceph-mon[30321]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:32.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:32 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:32.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:32 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:32.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:32 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:33 smithi184 ceph-mon[33536]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:33.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:33 smithi178 ceph-mon[39811]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:33.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:33 smithi178 ceph-mon[30321]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:35.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:35 smithi184 ceph-mon[33536]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:35.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:35 smithi178 ceph-mon[39811]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:35.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:35 smithi178 ceph-mon[30321]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:44:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:44:37.463 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:37 smithi184 ceph-mon[33536]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:37.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:37 smithi178 ceph-mon[39811]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:37.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:37 smithi178 ceph-mon[30321]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:39.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:39 smithi184 ceph-mon[33536]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:39.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:39 smithi178 ceph-mon[39811]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:39.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:39 smithi178 ceph-mon[30321]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:40.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:40 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:40.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:40 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:40.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:40 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:41.220 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:44:40 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:44:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:41 smithi184 ceph-mon[33536]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:41.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:41 smithi178 ceph-mon[39811]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:41.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:41 smithi178 ceph-mon[30321]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:42.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:42 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:42.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:42 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:42.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:42 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr:Note: switching to '8727ce939fc044a98f215e29696bb1433f5758d1'. 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr:state without impacting any branches by switching back to a branch. 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr: git switch -c 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-02-14T17:44:42.870 INFO:tasks.workunit.client.0.smithi178.stderr:Or undo this operation with: 2024-02-14T17:44:42.871 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-02-14T17:44:42.871 INFO:tasks.workunit.client.0.smithi178.stderr: git switch - 2024-02-14T17:44:42.871 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-02-14T17:44:42.871 INFO:tasks.workunit.client.0.smithi178.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-14T17:44:42.871 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-02-14T17:44:42.871 INFO:tasks.workunit.client.0.smithi178.stderr:HEAD is now at 8727ce939fc Merge branch 'node-proxy-handle-fetch-oob-error' of https://github.com/guits/ceph into wip-guits-testing-2024-02-14-1033 2024-02-14T17:44:42.873 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-14T17:44:42.911 INFO:tasks.workunit.client.0.smithi178.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-14T17:44:42.913 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-14T17:44:42.913 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-14T17:44:43.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:43 smithi184 ceph-mon[33536]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:43.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:43 smithi178 ceph-mon[30321]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:43.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:43 smithi178 ceph-mon[39811]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:43.743 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-14T17:44:43.804 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-14T17:44:43.844 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-14T17:44:43.846 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-14T17:44:43.846 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-14T17:44:43.885 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-14T17:44:43.888 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-02-14T17:44:43.888 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-14T17:44:43.944 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-14T17:44:43.945 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-14T17:44:43.945 DEBUG:teuthology.orchestra.run.smithi178:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8727ce939fc044a98f215e29696bb1433f5758d1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-02-14T17:44:44.011 INFO:tasks.workunit.client.0.smithi178.stderr:+ parallel=1 2024-02-14T17:44:44.011 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' '' = --serial ']' 2024-02-14T17:44:44.011 INFO:tasks.workunit.client.0.smithi178.stderr:+ crimson=0 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' '' = --crimson ']' 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:+ color= 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -t 1 ']' 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:+ GTEST_OUTPUT=xml:/home/ubuntu/cephtest/archive/gtest_xml_report 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:+ declare -A pids 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_aio 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_aio' 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_aio 2024-02-14T17:44:44.012 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.013 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_aio 2024-02-14T17:44:44.013 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80579 2024-02-14T17:44:44.013 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_aio on pid 80579' 2024-02-14T17:44:44.013 INFO:tasks.workunit.client.0.smithi178.stdout:test api_aio on pid 80579 2024-02-14T17:44:44.016 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80579 2024-02-14T17:44:44.016 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.016 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.016 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-02-14T17:44:44.016 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_aio_pp 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_aio_pp' 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_aio_pp 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.017 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.018 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_aio_pp 2024-02-14T17:44:44.019 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80586 2024-02-14T17:44:44.019 INFO:tasks.workunit.client.0.smithi178.stdout:test api_aio_pp on pid 80586 2024-02-14T17:44:44.023 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_aio_pp on pid 80586' 2024-02-14T17:44:44.023 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80586 2024-02-14T17:44:44.023 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-02-14T17:44:44.023 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_io 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_io' 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_io 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.024 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.025 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.026 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.027 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.027 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.027 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.027 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.027 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_io 2024-02-14T17:44:44.027 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80595 2024-02-14T17:44:44.027 INFO:tasks.workunit.client.0.smithi178.stdout:test api_io on pid 80595 2024-02-14T17:44:44.031 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_io on pid 80595' 2024-02-14T17:44:44.031 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80595 2024-02-14T17:44:44.031 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.031 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_io_pp 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_io_pp' 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_io_pp 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.032 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.033 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.034 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_io_pp 2024-02-14T17:44:44.035 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80606 2024-02-14T17:44:44.036 INFO:tasks.workunit.client.0.smithi178.stdout:test api_io_pp on pid 80606 2024-02-14T17:44:44.041 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_io_pp on pid 80606' 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80606 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_asio 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_asio' 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_asio 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.043 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.044 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.045 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.046 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.047 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.047 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.047 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.047 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.047 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_asio 2024-02-14T17:44:44.047 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80621 2024-02-14T17:44:44.047 INFO:tasks.workunit.client.0.smithi178.stdout:test api_asio on pid 80621 2024-02-14T17:44:44.053 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_asio on pid 80621' 2024-02-14T17:44:44.053 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80621 2024-02-14T17:44:44.053 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.053 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_list 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_list' 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_list 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.054 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.055 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.056 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.057 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.058 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.059 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80579/exe 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.060 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.061 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.061 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.061 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_list 2024-02-14T17:44:44.061 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80639 2024-02-14T17:44:44.061 INFO:tasks.workunit.client.0.smithi178.stdout:test api_list on pid 80639 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_list on pid 80639' 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80639 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_lock 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_lock' 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.066 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.067 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_lock 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.068 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.069 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.070 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_aio.log 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_aio: /' 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.071 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.072 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80586/exe 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.073 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.074 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.074 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.074 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.074 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.074 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.074 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_lock 2024-02-14T17:44:44.074 INFO:tasks.workunit.client.0.smithi178.stdout:test api_lock on pid 80659 2024-02-14T17:44:44.081 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80659 2024-02-14T17:44:44.081 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_lock on pid 80659' 2024-02-14T17:44:44.081 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80659 2024-02-14T17:44:44.081 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.081 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.081 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-02-14T17:44:44.081 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.081 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.081 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.082 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_lock_pp 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.083 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_lock_pp' 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.084 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.085 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.086 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_lock_pp 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_aio_pp: /' 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.087 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80595/exe 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.088 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.089 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.090 INFO:tasks.workunit.client.0.smithi178.stdout:test api_lock_pp on pid 80680 2024-02-14T17:44:44.093 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.093 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.093 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.093 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.093 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.093 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.093 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.093 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_lock_pp 2024-02-14T17:44:44.093 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.093 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80680 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_lock_pp on pid 80680' 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80680 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.094 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.095 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80606/exe 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_io.log 2024-02-14T17:44:44.096 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_io: /' 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_misc 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.097 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_misc' 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.098 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_misc 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_io_pp: /' 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.099 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.100 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.101 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.102 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.102 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.102 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.102 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.102 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.102 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.102 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.102 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-02-14T17:44:44.102 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80621/exe 2024-02-14T17:44:44.102 INFO:tasks.workunit.client.0.smithi178.stdout:test api_misc on pid 80705 2024-02-14T17:44:44.104 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_misc 2024-02-14T17:44:44.104 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80705 2024-02-14T17:44:44.104 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_misc on pid 80705' 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80705 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.105 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_misc_pp 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_misc_pp' 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.106 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_misc_pp 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.107 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.108 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_asio.log 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.109 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_asio: /' 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2024-02-14T17:44:44.110 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_misc_pp 2024-02-14T17:44:44.111 INFO:tasks.workunit.client.0.smithi178.stdout:test api_misc_pp on pid 80722 2024-02-14T17:44:44.113 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80722 2024-02-14T17:44:44.113 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_misc_pp on pid 80722' 2024-02-14T17:44:44.113 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80722 2024-02-14T17:44:44.113 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.113 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.113 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-02-14T17:44:44.113 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_tier_pp 2024-02-14T17:44:44.113 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-02-14T17:44:44.113 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_tier_pp' 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.114 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.115 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_tier_pp 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.116 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.117 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.117 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.117 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.117 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.117 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.117 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.117 INFO:tasks.workunit.client.0.smithi178.stdout:test api_tier_pp on pid 80734 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_tier_pp 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80734 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_tier_pp on pid 80734' 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80734 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_pool 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_pool' 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_pool 2024-02-14T17:44:44.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.122 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.123 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.124 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.125 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.126 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.127 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80639/exe 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.128 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.129 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.130 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.131 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_list.log 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.132 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_list: /' 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.133 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.134 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.135 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80722/exe 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.136 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.137 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.138 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.138 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2024-02-14T17:44:44.138 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_misc_pp: /' 2024-02-14T17:44:44.138 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80659/exe 2024-02-14T17:44:44.138 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.138 INFO:tasks.workunit.client.0.smithi178.stdout:test api_pool on pid 80779 2024-02-14T17:44:44.142 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_pool 2024-02-14T17:44:44.142 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80779 2024-02-14T17:44:44.142 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_pool on pid 80779' 2024-02-14T17:44:44.142 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80779 2024-02-14T17:44:44.142 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.142 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.142 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_snapshots 2024-02-14T17:44:44.142 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.142 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.142 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.143 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_snapshots' 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.144 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_lock: /' 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_lock.log 2024-02-14T17:44:44.145 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80680/exe 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.146 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_lock_pp: /' 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.147 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.148 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.148 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.148 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.148 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2024-02-14T17:44:44.148 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.148 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_snapshots 2024-02-14T17:44:44.148 INFO:tasks.workunit.client.0.smithi178.stdout:test api_snapshots on pid 80834 2024-02-14T17:44:44.156 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_snapshots 2024-02-14T17:44:44.156 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80834 2024-02-14T17:44:44.156 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_snapshots on pid 80834' 2024-02-14T17:44:44.159 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80834 2024-02-14T17:44:44.159 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.159 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.160 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_snapshots_pp 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_snapshots_pp' 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.161 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.162 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.163 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.164 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_snapshots_pp 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.165 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.166 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.167 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.168 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.169 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.170 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.170 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.170 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.170 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.170 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.170 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.170 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80705/exe 2024-02-14T17:44:44.170 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.170 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_snapshots_pp 2024-02-14T17:44:44.170 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80892 2024-02-14T17:44:44.170 INFO:tasks.workunit.client.0.smithi178.stdout:test api_snapshots_pp on pid 80892 2024-02-14T17:44:44.177 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_snapshots_pp on pid 80892' 2024-02-14T17:44:44.177 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80892 2024-02-14T17:44:44.177 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.177 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.177 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.177 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.177 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.177 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.177 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_misc: /' 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80734/exe 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_misc.log 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.178 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.179 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.180 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_stat 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_stat' 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.181 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.182 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.183 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.184 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_tier_pp: /' 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-02-14T17:44:44.185 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.186 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.187 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.188 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.189 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80834/exe 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.190 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.191 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_snapshots: /' 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_stat 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_stat 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.192 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80964 2024-02-14T17:44:44.193 INFO:tasks.workunit.client.0.smithi178.stdout:test api_stat on pid 80964 2024-02-14T17:44:44.197 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_stat on pid 80964' 2024-02-14T17:44:44.197 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=80964 2024-02-14T17:44:44.197 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.197 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.197 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80892/exe 2024-02-14T17:44:44.197 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-02-14T17:44:44.197 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.197 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.197 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.197 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.198 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_stat_pp 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_stat_pp' 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.199 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.200 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_stat_pp 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80779/exe 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.201 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.202 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2024-02-14T17:44:44.203 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_pool: /' 2024-02-14T17:44:44.204 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_stat_pp 2024-02-14T17:44:44.204 INFO:tasks.workunit.client.0.smithi178.stdout:test api_stat_pp on pid 81015 2024-02-14T17:44:44.205 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81015 2024-02-14T17:44:44.205 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_stat_pp on pid 81015' 2024-02-14T17:44:44.205 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81015 2024-02-14T17:44:44.205 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.205 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.205 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-02-14T17:44:44.205 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.205 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_watch_notify 2024-02-14T17:44:44.206 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_watch_notify' 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_watch_notify 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.207 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.208 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.208 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.208 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.208 INFO:tasks.workunit.client.0.smithi178.stdout:test api_watch_notify on pid 81024 2024-02-14T17:44:44.210 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_watch_notify 2024-02-14T17:44:44.210 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.210 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81024 2024-02-14T17:44:44.210 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_watch_notify on pid 81024' 2024-02-14T17:44:44.210 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81024 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_pool.log 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.211 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_watch_notify_pp 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_watch_notify_pp' 2024-02-14T17:44:44.212 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_watch_notify_pp 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.213 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.214 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.215 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.216 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.216 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.216 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_watch_notify_pp 2024-02-14T17:44:44.216 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81037 2024-02-14T17:44:44.216 INFO:tasks.workunit.client.0.smithi178.stdout:test api_watch_notify_pp on pid 81037 2024-02-14T17:44:44.218 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_watch_notify_pp on pid 81037' 2024-02-14T17:44:44.218 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81037 2024-02-14T17:44:44.218 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.218 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.218 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-02-14T17:44:44.218 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.218 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.218 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_cmd 2024-02-14T17:44:44.218 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.218 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.218 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_cmd' 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.219 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.220 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_cmd 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.221 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.222 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.222 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.222 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.222 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.222 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.222 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_cmd 2024-02-14T17:44:44.222 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81049 2024-02-14T17:44:44.222 INFO:tasks.workunit.client.0.smithi178.stdout:test api_cmd on pid 81049 2024-02-14T17:44:44.231 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_cmd on pid 81049' 2024-02-14T17:44:44.231 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81049 2024-02-14T17:44:44.231 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.231 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.231 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-02-14T17:44:44.231 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.231 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.232 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.233 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.234 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.235 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.236 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.237 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.238 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/80964/exe 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81015/exe 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_cmd_pp 2024-02-14T17:44:44.239 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_cmd_pp' 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.240 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_stat_pp: /' 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.241 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.242 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.243 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_stat.log 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_stat: /' 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.244 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.245 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.246 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_cmd_pp 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.247 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.248 INFO:tasks.workunit.client.0.smithi178.stdout:test api_cmd_pp on pid 81136 2024-02-14T17:44:44.251 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_cmd_pp 2024-02-14T17:44:44.251 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81136 2024-02-14T17:44:44.251 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_cmd_pp on pid 81136' 2024-02-14T17:44:44.251 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.251 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81136 2024-02-14T17:44:44.251 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.251 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81037/exe 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_service 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_service' 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_service 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.252 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.253 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.254 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.255 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.256 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.256 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.256 INFO:tasks.workunit.client.0.smithi178.stdout:test api_service on pid 81151 2024-02-14T17:44:44.259 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_service 2024-02-14T17:44:44.259 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81151 2024-02-14T17:44:44.259 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_service on pid 81151' 2024-02-14T17:44:44.259 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81151 2024-02-14T17:44:44.259 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_service_pp 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_service_pp' 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_service_pp 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.260 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-02-14T17:44:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.264 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.264 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.264 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81024/exe 2024-02-14T17:44:44.264 INFO:tasks.workunit.client.0.smithi178.stdout:test api_service_pp on pid 81179 2024-02-14T17:44:44.266 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_service_pp 2024-02-14T17:44:44.266 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81179 2024-02-14T17:44:44.266 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_service_pp on pid 81179' 2024-02-14T17:44:44.266 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81179 2024-02-14T17:44:44.266 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_c_write_operations 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_c_write_operations' 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.267 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.268 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_watch_notify: /' 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_c_write_operations 2024-02-14T17:44:44.269 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.270 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.271 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.271 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.271 INFO:tasks.workunit.client.0.smithi178.stdout:test api_c_write_operations on pid 81201 2024-02-14T17:44:44.274 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_c_write_operations 2024-02-14T17:44:44.274 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81201 2024-02-14T17:44:44.274 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_c_write_operations on pid 81201' 2024-02-14T17:44:44.274 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81201 2024-02-14T17:44:44.274 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.274 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_c_read_operations 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_c_read_operations' 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.275 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.276 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.277 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.278 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.279 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.280 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_c_read_operations 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.281 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.282 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.282 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.282 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.282 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.282 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.282 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.282 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.282 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.282 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.282 INFO:tasks.workunit.client.0.smithi178.stdout:test api_c_read_operations on pid 81229 2024-02-14T17:44:44.289 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_c_read_operations 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81229 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_c_read_operations on pid 81229' 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81229 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.291 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.292 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.293 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81136/exe 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s api_cls_remote_reads 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' api_cls_remote_reads' 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81151/exe 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo api_cls_remote_reads 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2024-02-14T17:44:44.301 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.302 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.303 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_service: /' 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81049/exe 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.304 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_service.log 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.306 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.306 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.306 INFO:tasks.workunit.client.0.smithi178.stdout:test api_cls_remote_reads on pid 81281 2024-02-14T17:44:44.308 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=api_cls_remote_reads 2024-02-14T17:44:44.308 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81281 2024-02-14T17:44:44.308 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test api_cls_remote_reads on pid 81281' 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81281 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s list_parallel 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_cmd: /' 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.309 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' list_parallel' 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-02-14T17:44:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_cmd.log 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo list_parallel 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.311 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.313 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.314 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.314 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.314 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.314 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.314 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.314 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=list_parallel 2024-02-14T17:44:44.314 INFO:tasks.workunit.client.0.smithi178.stdout:test list_parallel on pid 81314 2024-02-14T17:44:44.316 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81314 2024-02-14T17:44:44.316 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test list_parallel on pid 81314' 2024-02-14T17:44:44.316 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81314 2024-02-14T17:44:44.316 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.316 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.316 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.316 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.316 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.316 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s open_pools_parallel 2024-02-14T17:44:44.316 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' open_pools_parallel' 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.319 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo open_pools_parallel 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.321 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81179/exe 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.322 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.323 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.323 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.323 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.323 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.323 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.323 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.323 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.323 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.323 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.324 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.324 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.324 INFO:tasks.workunit.client.0.smithi178.stdout:test open_pools_parallel on pid 81335 2024-02-14T17:44:44.327 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.327 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.327 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=open_pools_parallel 2024-02-14T17:44:44.328 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.328 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.328 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.328 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.328 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.328 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81335 2024-02-14T17:44:44.328 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.328 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test open_pools_parallel on pid 81335' 2024-02-14T17:44:44.328 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.328 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81335 2024-02-14T17:44:44.328 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.329 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s delete_pools_parallel 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_service_pp: /' 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' delete_pools_parallel' 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81229/exe 2024-02-14T17:44:44.330 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.331 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.332 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.333 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo delete_pools_parallel 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81201/exe 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.334 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.335 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2024-02-14T17:44:44.337 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-02-14T17:44:44.338 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-02-14T17:44:44.338 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.338 INFO:tasks.workunit.client.0.smithi178.stdout:test delete_pools_parallel on pid 81373 2024-02-14T17:44:44.344 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=delete_pools_parallel 2024-02-14T17:44:44.344 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81373 2024-02-14T17:44:44.344 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test delete_pools_parallel on pid 81373' 2024-02-14T17:44:44.344 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81373 2024-02-14T17:44:44.345 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.345 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.345 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.345 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.345 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.345 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.345 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.346 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.346 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.346 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.346 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.346 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.346 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.346 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.347 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.347 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.347 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.347 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.347 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.347 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s cls 2024-02-14T17:44:44.347 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.347 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' cls' 2024-02-14T17:44:44.347 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-02-14T17:44:44.347 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.348 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.349 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.349 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.349 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.349 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.349 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.349 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.349 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.349 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.349 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.349 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.349 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.350 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.351 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.352 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.353 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.354 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.355 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.356 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81314/exe 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.357 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.358 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ list_parallel: /' 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.359 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.360 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.361 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.361 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.361 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.361 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.361 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.361 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.361 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.361 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.361 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.361 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.361 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.363 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.363 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.363 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.364 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.364 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.364 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.364 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.364 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.364 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.364 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.364 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.364 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_list_parallel.log 2024-02-14T17:44:44.364 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.365 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.365 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.365 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.365 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81335/exe 2024-02-14T17:44:44.365 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.365 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.365 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.365 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.365 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.366 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.366 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.366 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.366 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.366 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.366 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.366 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.366 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.367 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.367 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.367 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.367 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.367 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.367 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.367 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.367 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.368 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.368 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.368 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.368 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.368 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.368 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.368 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.369 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.369 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.369 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.369 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2024-02-14T17:44:44.369 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo cls 2024-02-14T17:44:44.369 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.370 INFO:tasks.workunit.client.0.smithi178.stdout:test cls on pid 81435 2024-02-14T17:44:44.373 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=cls 2024-02-14T17:44:44.373 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81435 2024-02-14T17:44:44.373 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test cls on pid 81435' 2024-02-14T17:44:44.373 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81435 2024-02-14T17:44:44.373 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.373 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s cmd 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' cmd' 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.374 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo cmd 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.375 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.376 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.376 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.376 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.376 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.376 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.376 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.376 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.376 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.376 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.376 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.376 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.377 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.378 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=cmd 2024-02-14T17:44:44.379 INFO:tasks.workunit.client.0.smithi178.stdout:test cmd on pid 81484 2024-02-14T17:44:44.384 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81484 2024-02-14T17:44:44.384 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test cmd on pid 81484' 2024-02-14T17:44:44.384 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81484 2024-02-14T17:44:44.384 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.384 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.384 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-02-14T17:44:44.384 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.384 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.384 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s handler_error 2024-02-14T17:44:44.384 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.385 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' handler_error' 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.386 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.387 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.387 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.387 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.387 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.387 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.387 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.387 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.387 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.387 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.387 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.387 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.388 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.388 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.388 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.388 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.388 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.388 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.388 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.388 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.389 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.389 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.389 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.389 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.389 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.389 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.389 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.389 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.390 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81281/exe 2024-02-14T17:44:44.390 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.390 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.390 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.390 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.390 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.390 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.391 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.391 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.391 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.391 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.391 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.391 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.391 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.392 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.392 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.392 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.392 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.392 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.392 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.392 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.393 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.393 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.393 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.393 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2024-02-14T17:44:44.393 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-02-14T17:44:44.393 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo handler_error 2024-02-14T17:44:44.393 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.394 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.394 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.394 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.394 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.394 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.394 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.394 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.394 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.394 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-02-14T17:44:44.395 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.396 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.396 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.396 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.396 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.396 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.396 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.396 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.396 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.396 INFO:tasks.workunit.client.0.smithi178.stdout:test handler_error on pid 81515 2024-02-14T17:44:44.399 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=handler_error 2024-02-14T17:44:44.399 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81515 2024-02-14T17:44:44.399 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test handler_error on pid 81515' 2024-02-14T17:44:44.399 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81515 2024-02-14T17:44:44.399 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.399 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.399 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.399 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.399 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.399 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81373/exe 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.400 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-02-14T17:44:44.401 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.402 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.403 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.404 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.404 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.404 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.404 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.404 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.404 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.404 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.404 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.404 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.404 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.404 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.405 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.406 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81484/exe 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.407 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_cmd.log 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_cmd 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ cmd: /' 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s io 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' io' 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo io 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.408 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.409 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.410 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.411 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.411 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.411 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.411 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.411 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.411 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.411 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.411 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.411 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.411 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.411 INFO:tasks.workunit.client.0.smithi178.stdout:test io on pid 81596 2024-02-14T17:44:44.415 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=io 2024-02-14T17:44:44.415 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81596 2024-02-14T17:44:44.415 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test io on pid 81596' 2024-02-14T17:44:44.415 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81596 2024-02-14T17:44:44.415 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.415 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.415 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.415 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.416 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s ec_io 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' ec_io' 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.417 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.418 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.419 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.420 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81435/exe 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.421 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.422 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_cls 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.423 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_cls.log 2024-02-14T17:44:44.424 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ cls: /' 2024-02-14T17:44:44.424 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.424 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.424 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.424 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.424 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.424 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.424 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.424 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.424 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.424 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.425 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo ec_io 2024-02-14T17:44:44.426 INFO:tasks.workunit.client.0.smithi178.stdout:test ec_io on pid 81682 2024-02-14T17:44:44.428 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=ec_io 2024-02-14T17:44:44.428 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81682 2024-02-14T17:44:44.428 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test ec_io on pid 81682' 2024-02-14T17:44:44.428 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81682 2024-02-14T17:44:44.428 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.428 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.428 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81515/exe 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s list 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.429 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.430 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.430 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' list' 2024-02-14T17:44:44.430 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.430 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.430 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.430 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.430 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.430 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.430 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.430 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.431 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ handler_error: /' 2024-02-14T17:44:44.432 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_handler_error 2024-02-14T17:44:44.432 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.432 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_handler_error.log 2024-02-14T17:44:44.432 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.432 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.432 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.432 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.432 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.432 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.432 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.433 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.434 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.434 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.434 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.434 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.434 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo list 2024-02-14T17:44:44.434 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.434 INFO:tasks.workunit.client.0.smithi178.stdout:test list on pid 81719 2024-02-14T17:44:44.436 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=list 2024-02-14T17:44:44.436 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81719 2024-02-14T17:44:44.436 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test list on pid 81719' 2024-02-14T17:44:44.436 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81719 2024-02-14T17:44:44.436 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.436 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.436 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s ec_list 2024-02-14T17:44:44.436 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' ec_list' 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.437 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo ec_list 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.438 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.439 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.439 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.439 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.439 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.439 INFO:tasks.workunit.client.0.smithi178.stdout:test ec_list on pid 81726 2024-02-14T17:44:44.444 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=ec_list 2024-02-14T17:44:44.444 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81726 2024-02-14T17:44:44.444 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test ec_list on pid 81726' 2024-02-14T17:44:44.444 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.444 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81726 2024-02-14T17:44:44.444 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.444 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.444 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.445 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.446 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s misc 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.447 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' misc' 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.448 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.449 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.450 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.451 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.452 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.453 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo misc 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.454 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.455 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.456 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.457 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.457 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.457 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.457 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.457 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.457 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.457 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.457 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.457 INFO:tasks.workunit.client.0.smithi178.stdout:test misc on pid 81778 2024-02-14T17:44:44.462 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.462 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.462 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=misc 2024-02-14T17:44:44.462 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.462 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.462 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81778 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test misc on pid 81778' 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81778 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.463 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.464 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s pool 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81682/exe 2024-02-14T17:44:44.465 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' pool' 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.466 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.467 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.468 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.469 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.470 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ ec_io: /' 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo pool 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_ec_io 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_ec_io.log 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.471 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.472 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.472 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.472 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.472 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.472 INFO:tasks.workunit.client.0.smithi178.stdout:test pool on pid 81831 2024-02-14T17:44:44.479 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.479 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=pool 2024-02-14T17:44:44.479 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.479 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81831 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test pool on pid 81831' 2024-02-14T17:44:44.480 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81831 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.481 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81719/exe 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.482 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.483 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.484 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.485 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s read_operations 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.486 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' read_operations' 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.487 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo read_operations 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81726/exe 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.488 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.489 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.489 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.489 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.489 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.489 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.489 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.489 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.489 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.489 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.489 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.489 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.490 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81596/exe 2024-02-14T17:44:44.490 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.490 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.490 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.490 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.490 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.490 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.490 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.490 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.490 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.490 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ list: /' 2024-02-14T17:44:44.491 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.491 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.491 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.491 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.491 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.491 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.491 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.491 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_list.log 2024-02-14T17:44:44.492 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_list 2024-02-14T17:44:44.492 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.492 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.492 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.492 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.492 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.492 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.492 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.493 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.493 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.493 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.493 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.493 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.493 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.493 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ io: /' 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_ec_list 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.494 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.495 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.496 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.496 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.496 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_io 2024-02-14T17:44:44.496 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_io.log 2024-02-14T17:44:44.496 INFO:tasks.workunit.client.0.smithi178.stdout:test read_operations on pid 81864 2024-02-14T17:44:44.498 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=read_operations 2024-02-14T17:44:44.498 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81864 2024-02-14T17:44:44.498 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test read_operations on pid 81864' 2024-02-14T17:44:44.498 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81864 2024-02-14T17:44:44.498 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.498 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.498 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ ec_list: /' 2024-02-14T17:44:44.498 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_ec_list.log 2024-02-14T17:44:44.498 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s snapshots 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' snapshots' 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo snapshots 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.499 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.500 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.501 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.501 INFO:tasks.workunit.client.0.smithi178.stdout:test snapshots on pid 81874 2024-02-14T17:44:44.508 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.508 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.508 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.508 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.508 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=snapshots 2024-02-14T17:44:44.508 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.508 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81874 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test snapshots on pid 81874' 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81874 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s watch_notify 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' watch_notify' 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.509 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.510 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo watch_notify 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.511 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.512 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.512 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.512 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.512 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.512 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.512 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.512 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.512 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.512 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.512 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.512 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.513 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.514 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81778/exe 2024-02-14T17:44:44.515 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.516 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.517 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.518 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81831/exe 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_misc.log 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.519 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.520 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_pool 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.521 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ pool: /' 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.522 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.523 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_misc 2024-02-14T17:44:44.524 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ misc: /' 2024-02-14T17:44:44.525 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.525 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.525 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.525 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.525 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.525 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_pool.log 2024-02-14T17:44:44.525 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.525 INFO:tasks.workunit.client.0.smithi178.stdout:test watch_notify on pid 81917 2024-02-14T17:44:44.528 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=watch_notify 2024-02-14T17:44:44.528 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81917 2024-02-14T17:44:44.528 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test watch_notify on pid 81917' 2024-02-14T17:44:44.528 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81917 2024-02-14T17:44:44.528 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:44:44.528 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:++ printf %25s write_operations 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:+ r=' write_operations' 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.529 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:++ echo write_operations 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:++ awk '{print $1}' 2024-02-14T17:44:44.530 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.531 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.532 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81864/exe 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:+ ff=write_operations 2024-02-14T17:44:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.534 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.534 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.534 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.534 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.534 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.534 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.534 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81941 2024-02-14T17:44:44.534 INFO:tasks.workunit.client.0.smithi178.stdout:test write_operations on pid 81941 2024-02-14T17:44:44.549 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'test write_operations on pid 81941' 2024-02-14T17:44:44.549 INFO:tasks.workunit.client.0.smithi178.stderr:+ pids[$f]=81941 2024-02-14T17:44:44.549 INFO:tasks.workunit.client.0.smithi178.stderr:+ ret=0 2024-02-14T17:44:44.549 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:44:44.549 INFO:tasks.workunit.client.0.smithi178.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-02-14T17:44:44.549 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:44:44.549 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80734 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80734 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:+ . /etc/bashrc 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:++ shopt -q login_shell 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:+++ umask 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.550 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.551 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:++ SHELL=/bin/bash 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.552 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:44:44.553 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.553 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.553 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:44:44.553 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.553 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.553 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:44:44.553 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.553 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.553 INFO:tasks.workunit.client.0.smithi178.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:44:44.553 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.553 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.554 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:44:44.554 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.554 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:44:44.554 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.554 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.554 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.554 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.554 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.554 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.554 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_read_operations 2024-02-14T17:44:44.554 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.555 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.555 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.555 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.555 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.555 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.555 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.555 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.555 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_read_operations.log 2024-02-14T17:44:44.555 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ read_operations: /' 2024-02-14T17:44:44.555 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.556 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.556 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.556 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.556 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.556 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.556 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.556 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.557 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.557 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:44:44.557 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:44:44.557 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:44:44.557 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.557 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:44:44.557 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.558 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:44:44.558 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:44:44.558 INFO:tasks.workunit.client.0.smithi178.stderr:+++ return 2024-02-14T17:44:44.558 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.558 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:44:44.558 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.559 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:44:44.559 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export S_COLORS=auto 2024-02-14T17:44:44.559 INFO:tasks.workunit.client.0.smithi178.stderr:+++ S_COLORS=auto 2024-02-14T17:44:44.559 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.559 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:44:44.559 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.559 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:44:44.560 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.560 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.560 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.560 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.560 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.560 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.561 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.562 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.563 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.564 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.565 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81874/exe 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.566 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.567 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.568 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81917/exe 2024-02-14T17:44:44.569 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/locale 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_snapshots 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.570 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_snapshots.log 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ snapshots: /' 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.571 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_watch_notify 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ watch_notify: /' 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:44:44.572 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:+++ unset LANG_backup config 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n '' ']' 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:44:44.573 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '' ']' 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:+++++ readlink /proc/81941/exe 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:++++ basename /usr/bin/bash 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_declare='declare -f' 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:+++ which_opt=-f 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export which_declare 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:+++ export -f which 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset i 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset -f pathmunge 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:44:44.574 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-02-14T17:44:44.575 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:44:44.575 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset rc 2024-02-14T17:44:44.575 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph_test_neorados_write_operations 2024-02-14T17:44:44.575 INFO:tasks.workunit.client.0.smithi178.stderr:+ tee ceph_test_neorados_write_operations.log 2024-02-14T17:44:44.575 INFO:tasks.workunit.client.0.smithi178.stderr:+ sed 's/^/ write_operations: /' 2024-02-14T17:44:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/408675295' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-14T17:44:45.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:45.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/408675295' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-14T17:44:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/408675295' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-14T17:44:46.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:44:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:44:49.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2145306142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi178-80663-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:49.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi178-81859-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:49.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi178-81815-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:49.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:48 smithi184 ceph-mon[33536]: osdmap e75: 8 total, 8 up, 8 in 2024-02-14T17:44:49.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2145306142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi178-80663-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:49.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi178-81859-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:49.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi178-81815-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:49.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:48 smithi178 ceph-mon[39811]: osdmap e75: 8 total, 8 up, 8 in 2024-02-14T17:44:49.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2145306142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi178-80663-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:49.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi178-81859-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:49.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi178-81815-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:49.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:48 smithi178 ceph-mon[30321]: osdmap e75: 8 total, 8 up, 8 in 2024-02-14T17:44:51.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:44:50 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:44:51.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-02-14T17:44:51.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [----------] Global test environment set-up. 2024-02-14T17:44:51.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [----------] 12 tests from AsioRados 2024-02-14T17:44:51.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-02-14T17:44:51.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (3 ms) 2024-02-14T17:44:51.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-02-14T17:44:51.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (0 ms) 2024-02-14T17:44:51.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-02-14T17:44:51.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-02-14T17:44:51.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-02-14T17:44:51.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (6 ms) 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (2 ms) 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (1 ms) 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (0 ms) 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (1 ms) 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (0 ms) 2024-02-14T17:44:51.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-14T17:44:51.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (1 ms) 2024-02-14T17:44:51.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [----------] 12 tests from AsioRados (17 ms total) 2024-02-14T17:44:51.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: 2024-02-14T17:44:51.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [----------] Global test environment tear-down 2024-02-14T17:44:51.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (7858 ms total) 2024-02-14T17:44:51.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_asio: [ PASSED ] 12 tests. 2024-02-14T17:44:52.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2084135393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi178-80647-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi178-80753-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3047104654' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi178-80695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3425700854' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi178-80802-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/746983266' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi178-80684-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: osdmap e76: 8 total, 8 up, 8 in 2024-02-14T17:44:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: from='client.34661 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi178-80663-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: from='client.34697 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi178-80753-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:51 smithi184 ceph-mon[33536]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi178-80695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2084135393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi178-80647-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi178-80753-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3047104654' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi178-80695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3425700854' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi178-80802-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/746983266' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi178-80684-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: osdmap e76: 8 total, 8 up, 8 in 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: from='client.34661 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi178-80663-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: from='client.34697 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi178-80753-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[39811]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi178-80695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2084135393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi178-80647-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi178-80753-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3047104654' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi178-80695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3425700854' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi178-80802-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/746983266' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi178-80684-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: osdmap e76: 8 total, 8 up, 8 in 2024-02-14T17:44:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: from='client.34661 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi178-80663-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: from='client.34697 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi178-80753-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:51 smithi178 ceph-mon[30321]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi178-80695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: pgmap v67: 196 pgs: 35 creating+peering, 132 active+clean, 29 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:44:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: pgmap v69: 356 pgs: 35 creating+peering, 132 active+clean, 189 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2084135393' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi178-80647-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1937129161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2623993349' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi178-80903-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2765940660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi178-81353-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4098570037' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi178-80957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2142570336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1529594415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi178-81091-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3979684779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi178-81099-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/206168618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi178-80783-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2104636237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi178-80984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/316316170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/99609754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi178-81192-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: pgmap v71: 772 pgs: 35 creating+peering, 132 active+clean, 605 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/36557491' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: pgmap v72: 772 pgs: 386 creating+peering, 174 active+clean, 212 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1511705699' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: osdmap e77: 8 total, 8 up, 8 in 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:44:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi178-81859-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi178-81815-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.34814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.34661 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi178-80663-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.34697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi178-80753-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi178-80695-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3136725292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81245-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2241690208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: osdmap e78: 8 total, 8 up, 8 in 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi178-80802-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.34841 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24688 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi178-80684-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.34757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi178-81192-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi178-80903-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi178-81353-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi178-80957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi178-81091-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi178-81099-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi178-80783-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi178-80984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.24853 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.223 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:52 smithi184 ceph-mon[33536]: from='client.34898 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81245-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: pgmap v67: 196 pgs: 35 creating+peering, 132 active+clean, 29 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:44:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: pgmap v69: 356 pgs: 35 creating+peering, 132 active+clean, 189 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2084135393' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi178-80647-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1937129161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2623993349' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi178-80903-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2765940660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi178-81353-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4098570037' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi178-80957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2142570336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1529594415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi178-81091-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3979684779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi178-81099-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/206168618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi178-80783-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2104636237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi178-80984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/316316170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/99609754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi178-81192-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: pgmap v71: 772 pgs: 35 creating+peering, 132 active+clean, 605 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/36557491' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: pgmap v67: 196 pgs: 35 creating+peering, 132 active+clean, 29 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: pgmap v69: 356 pgs: 35 creating+peering, 132 active+clean, 189 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2084135393' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi178-80647-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1937129161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2623993349' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi178-80903-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2765940660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi178-81353-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4098570037' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi178-80957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2142570336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1529594415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi178-81091-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3979684779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi178-81099-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/206168618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi178-80783-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2104636237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi178-80984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/316316170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/99609754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi178-81192-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: pgmap v71: 772 pgs: 35 creating+peering, 132 active+clean, 605 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:44:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/36557491' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: pgmap v72: 772 pgs: 386 creating+peering, 174 active+clean, 212 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1511705699' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: osdmap e77: 8 total, 8 up, 8 in 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi178-81859-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi178-81815-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.34814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.34661 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi178-80663-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.34697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi178-80753-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi178-80695-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3136725292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81245-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2241690208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:53.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: osdmap e78: 8 total, 8 up, 8 in 2024-02-14T17:44:53.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi178-80802-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.34841 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24688 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi178-80684-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.34757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi178-81192-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi178-80903-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi178-81353-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi178-80957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi178-81091-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi178-81099-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi178-80783-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi178-80984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.24853 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[30321]: from='client.34898 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81245-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:44:53.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: pgmap v72: 772 pgs: 386 creating+peering, 174 active+clean, 212 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:44:53.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1511705699' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:44:53.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: osdmap e77: 8 total, 8 up, 8 in 2024-02-14T17:44:53.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:44:53.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi178-81859-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:53.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:53.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi178-81815-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:53.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.34814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.34661 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi178-80663-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.34697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi178-80753-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi178-80695-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3136725292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81245-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2241690208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: osdmap e78: 8 total, 8 up, 8 in 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi178-80802-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.34841 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81504-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24688 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi178-80684-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.34757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi178-81192-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi178-80903-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.340 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi178-81353-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.341 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi178-80957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.341 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.341 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi178-81091-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.341 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi178-81099-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.341 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi178-80783-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.341 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi178-80984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.341 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.24853 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.341 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:52 smithi178 ceph-mon[39811]: from='client.34898 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81245-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:53.494 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-02-14T17:44:53.494 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [----------] Global test environment set-up. 2024-02-14T17:44:53.494 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [----------] 4 tests from LibRadosService 2024-02-14T17:44:53.494 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-02-14T17:44:53.494 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5030 ms) 2024-02-14T17:44:53.494 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-02-14T17:44:53.494 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [ OK ] LibRadosService.RegisterLate (33 ms) 2024-02-14T17:44:53.494 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: cluster: 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: id: 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: health: HEALTH_OK 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: services: 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: data: 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: usage: 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:53.495 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: pgs: 132 active+clean 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: io: 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: cluster: 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: id: 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: health: HEALTH_OK 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: services: 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-14T17:44:53.496 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: data: 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: usage: 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: pgs: 132 active+clean 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: io: 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: cluster: 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: id: 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: health: HEALTH_WARN 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 19 pool(s) do not have an application enabled 2024-02-14T17:44:53.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: services: 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: data: 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: pools: 26 pools, 772 pgs 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: usage: 243 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: pgs: 27.461% pgs unknown 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 50.000% pgs not active 2024-02-14T17:44:53.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 386 creating+peering 2024-02-14T17:44:53.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 212 unknown 2024-02-14T17:44:53.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 174 active+clean 2024-02-14T17:44:53.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:44:53.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: io: 2024-02-14T17:44:53.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: client: 1.5 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-02-14T17:44:53.523 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-02-14T17:44:53.523 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [----------] Global test environment set-up. 2024-02-14T17:44:53.523 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [----------] 7 tests from LibRadosList 2024-02-14T17:44:53.523 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-02-14T17:44:53.523 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ OK ] LibRadosList.ListObjects (1397 ms) 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (30 ms) 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: myset foo1,foo2,foo3 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo1 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo2 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo3 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: myset foo1,foo4,foo5 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo4 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo5 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo1 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: myset foo6,foo7 2024-02-14T17:44:53.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo7 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo6 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: ns1:foo4 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: ns1:foo5 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: ns2:foo7 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: ns2:foo6 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: ns1:foo1 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: :foo1 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: :foo2 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: :foo3 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (78 ms) 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 1 0 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 10 0 2024-02-14T17:44:53.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 13 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 7 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 14 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 0 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 15 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 11 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 5 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 8 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 6 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 3 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 4 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 12 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 9 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 2 0 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (66 ms) 2024-02-14T17:44:53.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: x cursor=MIN 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=1 cursor=12:02547ec2:::1:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=10 cursor=12:52ea6a34:::10:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=13 cursor=12:566253c9:::13:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=7 cursor=12:5c6b0b28:::7:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=14 cursor=12:62a1935d:::14:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=0 cursor=12:6cac518f:::0:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=15 cursor=12:863748b0:::15:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=11 cursor=12:89d3ae78:::11:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=5 cursor=12:b29083e3:::5:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=8 cursor=12:bd63b0f1:::8:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=6 cursor=12:c4fdafeb:::6:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=3 cursor=12:cfc208b3:::3:head 2024-02-14T17:44:53.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=4 cursor=12:d83876eb:::4:head 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=12 cursor=12:de5d7c5f:::12:head 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=9 cursor=12:e960b815:::9:head 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > oid=2 cursor=12:f905c69b:::2:head 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: FIRST> seek to MIN oid=1 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=1 cursor=12:02547ec2:::1:head 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:02547ec2:::1:head 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:02547ec2:::1:head -> 1 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=10 cursor=12:52ea6a34:::10:head 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:52ea6a34:::10:head 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:52ea6a34:::10:head -> 10 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=13 cursor=12:566253c9:::13:head 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:566253c9:::13:head 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:566253c9:::13:head -> 13 2024-02-14T17:44:53.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=7 cursor=12:5c6b0b28:::7:head 2024-02-14T17:44:53.529 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:5c6b0b28:::7:head 2024-02-14T17:44:53.529 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:5c6b0b28:::7:head -> 7 2024-02-14T17:44:53.587 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=14 cursor=12:62a1935d:::14:head 2024-02-14T17:44:53.587 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:62a1935d:::14:head 2024-02-14T17:44:53.588 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:62a1935d:::14:head -> 14 2024-02-14T17:44:53.588 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=0 cursor=12:6cac518f:::0:head 2024-02-14T17:44:53.588 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:6cac518f:::0:head 2024-02-14T17:44:53.588 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:6cac518f:::0:head -> 0 2024-02-14T17:44:53.588 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=15 cursor=12:863748b0:::15:head 2024-02-14T17:44:53.588 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:863748b0:::15:head 2024-02-14T17:44:53.588 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:863748b0:::15:head -> 15 2024-02-14T17:44:53.588 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=11 cursor=12:89d3ae78:::11:head 2024-02-14T17:44:53.588 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:89d3ae78:::11:head 2024-02-14T17:44:53.588 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:89d3ae78:::11:head -> 11 2024-02-14T17:44:53.589 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=5 cursor=12:b29083e3:::5:head 2024-02-14T17:44:53.589 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:b29083e3:::5:head 2024-02-14T17:44:53.589 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:b29083e3:::5:head -> 5 2024-02-14T17:44:53.589 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=8 cursor=12:bd63b0f1:::8:head 2024-02-14T17:44:53.589 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:bd63b0f1:::8:head 2024-02-14T17:44:53.589 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:bd63b0f1:::8:head -> 8 2024-02-14T17:44:53.589 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=6 cursor=12:c4fdafeb:::6:head 2024-02-14T17:44:53.589 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:c4fdafeb:::6:head 2024-02-14T17:44:53.589 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:c4fdafeb:::6:head -> 6 2024-02-14T17:44:53.589 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=3 cursor=12:cfc208b3:::3:head 2024-02-14T17:44:53.589 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:cfc208b3:::3:head 2024-02-14T17:44:53.590 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:cfc208b3:::3:head -> 3 2024-02-14T17:44:53.590 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=4 cursor=12:d83876eb:::4:head 2024-02-14T17:44:53.590 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:d83876eb:::4:head 2024-02-14T17:44:53.590 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:d83876eb:::4:head -> 4 2024-02-14T17:44:53.590 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=12 cursor=12:de5d7c5f:::12:head 2024-02-14T17:44:53.590 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:de5d7c5f:::12:head 2024-02-14T17:44:53.590 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:de5d7c5f:::12:head -> 12 2024-02-14T17:44:53.590 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=9 cursor=12:e960b815:::9:head 2024-02-14T17:44:53.590 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:e960b815:::9:head 2024-02-14T17:44:53.590 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:e960b815:::9:head -> 9 2024-02-14T17:44:53.590 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : oid=2 cursor=12:f905c69b:::2:head 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:f905c69b:::2:head 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:f905c69b:::2:head -> 2 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:f905c69b:::2:head 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:f905c69b:::2:head expected=12:f905c69b:::2:head 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:f905c69b:::2:head -> 2 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=2 expected=2 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:bd63b0f1:::8:head 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:bd63b0f1:::8:head expected=12:bd63b0f1:::8:head 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:bd63b0f1:::8:head -> 8 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=8 expected=8 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:b29083e3:::5:head 2024-02-14T17:44:53.591 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:b29083e3:::5:head expected=12:b29083e3:::5:head 2024-02-14T17:44:53.592 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:b29083e3:::5:head -> 5 2024-02-14T17:44:53.592 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=5 expected=5 2024-02-14T17:44:53.592 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:89d3ae78:::11:head 2024-02-14T17:44:53.592 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:89d3ae78:::11:head expected=12:89d3ae78:::11:head 2024-02-14T17:44:53.592 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:89d3ae78:::11:head -> 11 2024-02-14T17:44:53.592 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=11 expected=11 2024-02-14T17:44:53.592 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:02547ec2:::1:head 2024-02-14T17:44:53.592 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:02547ec2:::1:head expected=12:02547ec2:::1:head 2024-02-14T17:44:53.592 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:02547ec2:::1:head -> 1 2024-02-14T17:44:53.592 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=1 expected=1 2024-02-14T17:44:53.592 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:e960b815:::9:head 2024-02-14T17:44:53.593 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:e960b815:::9:head expected=12:e960b815:::9:head 2024-02-14T17:44:53.593 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:e960b815:::9:head -> 9 2024-02-14T17:44:53.593 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=9 expected=9 2024-02-14T17:44:53.593 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:de5d7c5f:::12:head 2024-02-14T17:44:53.593 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:de5d7c5f:::12:head expected=12:de5d7c5f:::12:head 2024-02-14T17:44:53.593 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:de5d7c5f:::12:head -> 12 2024-02-14T17:44:53.593 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=12 expected=12 2024-02-14T17:44:53.593 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:d83876eb:::4:head 2024-02-14T17:44:53.593 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:d83876eb:::4:head expected=12:d83876eb:::4:head 2024-02-14T17:44:53.593 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:d83876eb:::4:head -> 4 2024-02-14T17:44:53.724 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=4 list_parallel: process_1_[81487]: starting. 2024-02-14T17:44:53.724 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_1_[81487]: creating pool ceph_test_rados_list_parallel.smithi178-81410 2024-02-14T17:44:53.724 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_1_[81487]: created object 0... 2024-02-14T17:44:53.724 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_1_[81487]: created object 25... 2024-02-14T17:44:53.724 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_1_[81487]: created object 49... 2024-02-14T17:44:53.724 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_1_[81487]: finishing. 2024-02-14T17:44:53.724 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_1_[81487]: shutting down. 2024-02-14T17:44:53.724 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_2_[81489]: starting. 2024-02-14T17:44:53.724 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_2_[81489]: listing objects. 2024-02-14T17:44:53.724 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_2_[81489]: listed object 0... 2024-02-14T17:44:53.724 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_2_[81489]: listed object 25... 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_2_[81489]: saw 50 objects 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_2_[81489]: shutting down. 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_3_[82658]: starting. 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_3_[82658]: creating pool ceph_test_rados_list_parallel.smithi178-81410 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_3_[82658]: created object 0... 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_3_[82658]: created object 25... 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_3_[82658]: created object 49... 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_3_[82658]: finishing. 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_3_[82658]: shutting down. 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_4_[82659]: starting. 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_4_[82659]: listing objects. 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_4_[82659]: listed object 0... 2024-02-14T17:44:53.725 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_4_[82659]: listed object 25... 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_4_[82659]: saw 43 objects 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_4_[82659]: shutting down. 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_5_[82660]: starting. 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_5_[82660]: removed 25 objects... 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_5_[82660]: removed half of the objects 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_5_[82660]: removed 50 objects... 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_5_[82660]: removed 50 objects 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_5_[82660]: shutting down. 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_6_[82706]: starting. 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_6_[82706]: creating pool ceph_test_rados_list_parallel.smithi178-81410 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_6_[82706]: created object 0... 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_6_[82706]: created object 25... 2024-02-14T17:44:53.726 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_6_[82706]: created object 49... 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_6_[82706]: finishing. 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_6_[82706]: shutting down. 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_7_[82707]: starting. 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_7_[82707]: listing objects. 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_7_[82707]: listed object 0... 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_7_[82707]: listed object 25... 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_7_[82707]: listed object 50... 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_7_[82707]: saw 56 objects 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_7_[82707]: shutting down. 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_8_[82708]: starting. 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_8_[82708]: added 25 objects... 2024-02-14T17:44:53.727 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_8_[82708]: added half of the objects 2024-02-14T17:44:53.728 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_8_[82708]: added 50 objects... 2024-02-14T17:44:53.728 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_8_[82708]: added 50 objects 2024-02-14T17:44:53.728 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_8_[82708]: shutting down. 2024-02-14T17:44:53.728 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.728 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.909 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.909 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_9_[82786]: starting. 2024-02-14T17:44:53.909 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_9_[82786]: creating pool ceph_test_rados_list_parallel.smithi178-81410 2024-02-14T17:44:53.909 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_9_[82786]: created object 0... 2024-02-14T17:44:53.909 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_9_[82786]: created object 25... 2024-02-14T17:44:53.909 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_9_[82786]: created object 49... 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_9_[82786]: finishing. 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_9_[82786]: shutting down. 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_10_[82787]: starting. 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_10_[82787]: listing objects. 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_10_[82787]: listed object 0... 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_10_[82787]: listed object 25... 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_10_[82787]: listed object 50... 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_10_[82787]: listed object 75... 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_10_[82787]: saw 97 objects 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_10_[82787]: shutting down. 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.910 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_13_[82790]: starting. 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_13_[82790]: removed 25 objects... 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_13_[82790]: removed half of the objects 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_13_[82790]: removed 50 objects... 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_13_[82790]: removed 50 objects 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_13_[82790]: shutting down. 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_11_[82788]: starting. 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_11_[82788]: added 25 objects... 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_11_[82788]: added half of the objects 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_11_[82788]: added 50 objects... 2024-02-14T17:44:53.911 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_11_[82788]: added 50 objects 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_11_[82788]: shutting down. 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_12_[82789]: starting. 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_12_[82789]: added 25 objects... 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_12_[82789]: added half of the objects 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_12_[82789]: added 50 objects... 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_12_[82789]: added 50 objects 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_12_[82789]: shutting down. 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.912 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_14_[82866]: starting. 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_14_[82866]: creating pool ceph_test_rados_list_parallel.smithi178-81410 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_14_[82866]: created object 0... 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_14_[82866]: created object 25... 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_14_[82866]: created object 49... 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_14_[82866]: finishing. 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_14_[82866]: shutting down. 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_15_[82867]: starting. 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_15_[82867]: listing objects. 2024-02-14T17:44:53.913 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_15_[82867]: listed object 0... 2024-02-14T17:44:54.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:54.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi178-81859-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:44:54.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi178-81815-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:44:54.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.34814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi178-80802-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.34841 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81504-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24688 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi178-80684-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.34757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi178-81192-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi178-80903-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi178-81353-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi178-80957-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi178-81091-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi178-81099-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi178-80783-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi178-80984-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24853 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81278-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.34898 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81245-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:44:54.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: osdmap e79: 8 total, 8 up, 8 in 2024-02-14T17:44:54.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi178-81815-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:54.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi178-81146-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:54.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi178-81859-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:44:54.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi178-81815-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:54.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi178-81146-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:54.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi178-81859-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:44:54.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3032302781' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:44:54.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:54.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:54.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi178-81859-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:44:54.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi178-81815-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:44:54.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.34814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi178-80802-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.34841 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81504-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24688 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi178-80684-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.34757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi178-81192-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi178-80903-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi178-81353-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi178-80957-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi178-81091-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi178-81099-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi178-80783-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi178-80984-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:54.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi178-81859-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi178-81815-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.34814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi178-80802-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.34841 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81504-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24688 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi178-80684-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.34757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi178-81192-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi178-80903-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi178-81353-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi178-80957-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi178-81091-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi178-81099-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi178-80783-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi178-80984-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24853 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81278-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.34898 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81245-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:44:54.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: osdmap e79: 8 total, 8 up, 8 in 2024-02-14T17:44:54.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi178-81815-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:54.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi178-81146-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi178-81859-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi178-81815-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi178-81146-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi178-81859-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3032302781' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24853 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81278-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.34898 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81245-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: osdmap e79: 8 total, 8 up, 8 in 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi178-81815-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:54.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi178-81146-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:54.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi178-81859-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:44:54.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi178-81815-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:54.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi178-81146-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:44:54.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi178-81859-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:44:54.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3032302781' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:44:54.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:55.283 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_15_[82867]: listed object 2 api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-02-14T17:44:55.283 INFO:tasks.workunit.client.0.smithi178.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-02-14T17:44:55.283 INFO:tasks.workunit.client.0.smithi178.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-02-14T17:44:55.283 INFO:tasks.workunit.client.0.smithi178.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-02-14T17:44:55.283 INFO:tasks.workunit.client.0.smithi178.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (10925 ms) 2024-02-14T17:44:55.283 INFO:tasks.workunit.client.0.smithi178.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (10925 ms total) 2024-02-14T17:44:55.283 INFO:tasks.workunit.client.0.smithi178.stdout: api_cls_remote_reads: 2024-02-14T17:44:55.283 INFO:tasks.workunit.client.0.smithi178.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-02-14T17:44:55.283 INFO:tasks.workunit.client.0.smithi178.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (10925 ms total) 2024-02-14T17:44:55.284 INFO:tasks.workunit.client.0.smithi178.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-02-14T17:44:56.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:44:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:44:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:44:56.312 INFO:tasks.workunit.client.0.smithi178.stdout:5... 2024-02-14T17:44:56.312 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_15_[82867]: listed object 50... 2024-02-14T17:44:56.312 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_15_[82867]: listed object 75... 2024-02-14T17:44:56.312 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_15_[82867]: listed object 100... 2024-02-14T17:44:56.312 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_15_[82867]: listed object 125... 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_15_[82867]: saw 150 objects 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_15_[82867]: shutting down. 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_16_[82868]: starting. 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_16_[82868]: added 25 objects... 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_16_[82868]: added half of the objects 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_16_[82868]: added 50 objects... 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_16_[82868]: added 50 objects 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: process_16_[82868]: shutting down. 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:56.313 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:56.314 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:56.314 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******************************* 2024-02-14T17:44:56.314 INFO:tasks.workunit.client.0.smithi178.stdout: list_parallel: ******* SUCCESS ********** 2024-02-14T17:44:56.323 INFO:tasks.workunit.client.0.smithi178.stdout: handler_error: Running main() from gmock_main.cc 2024-02-14T17:44:56.323 INFO:tasks.workunit.client.0.smithi178.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-02-14T17:44:56.323 INFO:tasks.workunit.client.0.smithi178.stdout: handler_error: [----------] Global test environment set-up. 2024-02-14T17:44:56.323 INFO:tasks.workunit.client.0.smithi178.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-02-14T17:44:56.323 INFO:tasks.workunit.client.0.smithi178.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-02-14T17:44:56.323 INFO:tasks.workunit.client.0.smithi178.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (11921 ms) 2024-02-14T17:44:56.323 INFO:tasks.workunit.client.0.smithi178.stdout: handler_error: [----------] 1 test from neocls_handler_error (11921 ms total) 2024-02-14T17:44:56.323 INFO:tasks.workunit.client.0.smithi178.stdout: handler_error: 2024-02-14T17:44:56.323 INFO:tasks.workunit.client.0.smithi178.stdout: handler_error: [----------] Global test environment tear-down 2024-02-14T17:44:56.323 INFO:tasks.workunit.client.0.smithi178.stdout: handler_error: [==========] 1 test from 1 test suite ran. (11921 ms total) 2024-02-14T17:44:56.323 INFO:tasks.workunit.client.0.smithi178.stdout: handler_error: [ PASSED ] 1 test. 2024-02-14T17:44:56.339 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-02-14T17:44:56.339 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-02-14T17:44:56.339 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-02-14T17:44:56.339 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-02-14T17:44:56.339 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-02-14T17:44:56.339 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (75 ms) 2024-02-14T17:44:56.339 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-02-14T17:44:56.339 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (67 ms) 2024-02-14T17:44:56.339 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-02-14T17:44:56.339 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (11931 ms) 2024-02-14T17:44:56.340 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (12073 ms total) 2024-02-14T17:44:56.340 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: 2024-02-14T17:44:56.340 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-02-14T17:44:56.340 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (12073 ms total) 2024-02-14T17:44:56.340 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-02-14T17:44:56.359 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-02-14T17:44:56.359 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-02-14T17:44:56.359 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (305 ms) 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (9 ms) 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (7 ms) 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (27 ms) 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (9 ms) 2024-02-14T17:44:56.360 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (6 ms) 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (4 ms) 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (5 ms) 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (3 ms) 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (5 ms) 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (5 ms) 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-02-14T17:44:56.361 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (13 ms) 2024-02-14T17:44:56.362 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-02-14T17:44:56.362 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (11 ms) 2024-02-14T17:44:56.362 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-02-14T17:44:56.362 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (10 ms) 2024-02-14T17:44:56.362 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-02-14T17:44:56.362 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (4 ms) 2024-02-14T17:44:56.362 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (428 ms total) 2024-02-14T17:44:56.362 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: 2024-02-14T17:44:56.362 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-02-14T17:44:56.362 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (12085 ms total) 2024-02-14T17:44:56.362 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-02-14T17:44:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1682394948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:56 smithi178 ceph-mon[30321]: osdmap e80: 8 total, 8 up, 8 in 2024-02-14T17:44:56.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:56.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:56.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1682394948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:56.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:56 smithi178 ceph-mon[39811]: osdmap e80: 8 total, 8 up, 8 in 2024-02-14T17:44:56.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1682394948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:56 smithi184 ceph-mon[33536]: osdmap e80: 8 total, 8 up, 8 in 2024-02-14T17:44:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.401 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: Running main() from gmock_main.cc 2024-02-14T17:44:57.401 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [----------] Global test environment set-up. 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (52 ms) 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (69 ms) 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (11956 ms) 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:54.933611+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/1682394948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:54.933960+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.248104+0000 mon.a [INF] from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.248287+0000 mon.a [INF] from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.261825+0000 mon.a [INF] from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi178-81815-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi178-81815-1"}]': finished 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.261968+0000 mon.a [INF] from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi178-81146-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.262072+0000 mon.a [INF] from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi178-81859-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.281543+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.554710+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.555282+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.555702+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.556090+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.556561+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.556932+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.557309+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.557744+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.558261+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:55.558747+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:44:57.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.049890+0000 mon.a [INF] from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.293109+0000 mon.a [INF] from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:57.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.301720+0000 mon.a [INF] from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.301794+0000 mon.a [INF] from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.301850+0000 mon.a [INF] from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.309367+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/2378355378' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.318095+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:57.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.403757+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:57.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.433569+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:44:57.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.434771+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:44:57.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.441750+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:44:57.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.442438+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:44:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: pgmap v75: 1188 pgs: 364 creating+peering, 132 active+clean, 692 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:44:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi178-81815-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi178-81815-1"}]': finished 2024-02-14T17:44:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi178-81146-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:44:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi178-81859-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: pgmap v78: 1148 pgs: 36 creating+activating, 215 creating+peering, 420 active+clean, 477 unknown; 49 MiB data, 257 MiB used, 715 GiB / 715 GiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 108 op/s 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:44:57.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: osdmap e81: 8 total, 8 up, 8 in 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: 14.1 deep-scrub starts 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: 14.1 deep-scrub ok 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: osdmap e82: 8 total, 8 up, 8 in 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2378355378' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:57.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:57 smithi184 ceph-mon[33536]: from='client.35207 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: pgmap v75: 1188 pgs: 364 creating+peering, 132 active+clean, 692 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:44:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi178-81815-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi178-81815-1"}]': finished 2024-02-14T17:44:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi178-81146-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:44:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi178-81859-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:44:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: pgmap v78: 1148 pgs: 36 creating+activating, 215 creating+peering, 420 active+clean, 477 unknown; 49 MiB data, 257 MiB used, 715 GiB / 715 GiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 108 op/s 2024-02-14T17:44:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:44:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:44:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:44:57.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:44:57.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:44:57.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:44:57.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:44:57.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:44:57.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:44:57.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:44:57.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:44:57.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: pgmap v75: 1188 pgs: 364 creating+peering, 132 active+clean, 692 unknown; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi178-81815-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi178-81815-1"}]': finished 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi178-81146-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi178-81859-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: pgmap v78: 1148 pgs: 36 creating+activating, 215 creating+peering, 420 active+clean, 477 unknown; 49 MiB data, 257 MiB used, 715 GiB / 715 GiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 108 op/s 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:44:57.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:44:57.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: osdmap e81: 8 total, 8 up, 8 in 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: 14.1 deep-scrub starts 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: 14.1 deep-scrub ok 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: osdmap e82: 8 total, 8 up, 8 in 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2378355378' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:57.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:57.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:44:57.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:44:57.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:44:57.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:44:57.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:44:57.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:44:57.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:44:57.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:44:57.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:44:57.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:44:57.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: osdmap e81: 8 total, 8 up, 8 in 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: 14.1 deep-scrub starts 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: 14.1 deep-scrub ok 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi178-80647-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: osdmap e82: 8 total, 8 up, 8 in 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2378355378' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:44:57.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:44:57.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:44:57.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:44:57.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:57.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:57.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[39811]: from='client.35207 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:57.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:44:57.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:44:57.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:57.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:57.841 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:57 smithi178 ceph-mon[30321]: from='client.35207 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.442532+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.925296+0000 mon.a [INF] from='client.35207 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.396329+0000 mon.a [WRN] Health check update: 19 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.405607+0000 mon.a [INF] from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.406493+0000 mon.a [INF] from='client.35207 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.415050+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.418237+0000 mon.a [INF] from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.419038+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.419729+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/3480187384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.425428+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.427811+0000 mon.a [INF] from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.428141+0000 mon.a [INF] from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:58.634 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.430785+0000 mon.a [INF] from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:58.635 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.435045+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:58.635 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.440680+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:58.635 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:57.917805+0000 mon.a [INF] from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: 14.2 deep-scrub starts 2024-02-14T17:44:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: 14.2 deep-scrub ok 2024-02-14T17:44:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: 14.0 deep-scrub starts 2024-02-14T17:44:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: 14.0 deep-scrub ok 2024-02-14T17:44:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: 14.8 deep-scrub starts 2024-02-14T17:44:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: 14.8 deep-scrub ok 2024-02-14T17:44:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: 14.4 deep-scrub starts 2024-02-14T17:44:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: 14.4 deep-scrub ok 2024-02-14T17:44:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: pgmap v80: 724 pgs: 48 creating+activating, 71 creating+peering, 451 active+clean, 154 unknown; 121 MiB data, 445 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 27 MiB/s wr, 491 op/s 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: 14.3 deep-scrub starts 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: 14.3 deep-scrub ok 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: Health check update: 19 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.35207 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: osdmap e83: 8 total, 8 up, 8 in 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3480187384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:58.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:58.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:58.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:58.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:58.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:58 smithi184 ceph-mon[33536]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: 14.2 deep-scrub starts 2024-02-14T17:44:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: 14.2 deep-scrub ok 2024-02-14T17:44:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: 14.0 deep-scrub starts 2024-02-14T17:44:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: 14.0 deep-scrub ok 2024-02-14T17:44:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: 14.8 deep-scrub starts 2024-02-14T17:44:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: 14.8 deep-scrub ok 2024-02-14T17:44:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: 14.4 deep-scrub starts 2024-02-14T17:44:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: 14.4 deep-scrub ok 2024-02-14T17:44:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: pgmap v80: 724 pgs: 48 creating+activating, 71 creating+peering, 451 active+clean, 154 unknown; 121 MiB data, 445 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 27 MiB/s wr, 491 op/s 2024-02-14T17:44:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: 14.3 deep-scrub starts 2024-02-14T17:44:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: 14.3 deep-scrub ok 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: Health check update: 19 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.35207 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: osdmap e83: 8 total, 8 up, 8 in 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3480187384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:59.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[39811]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: 14.2 deep-scrub starts 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: 14.2 deep-scrub ok 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: 14.0 deep-scrub starts 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: 14.0 deep-scrub ok 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: 14.8 deep-scrub starts 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: 14.8 deep-scrub ok 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: 14.4 deep-scrub starts 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: 14.4 deep-scrub ok 2024-02-14T17:44:59.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: pgmap v80: 724 pgs: 48 creating+activating, 71 creating+peering, 451 active+clean, 154 unknown; 121 MiB data, 445 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 27 MiB/s wr, 491 op/s 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: 14.3 deep-scrub starts 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: 14.3 deep-scrub ok 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: Health check update: 19 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.35207 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: osdmap e83: 8 total, 8 up, 8 in 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3480187384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:44:59.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:59.087 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:44:59.087 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:58 smithi178 ceph-mon[30321]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:59.646 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.154088+0000 mon.a [INF] from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:44:59.646 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:56.444163+0000 client.admin [INF] onexx 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.637958+0000 mon.a [INF] from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.638242+0000 mon.a [INF] from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.638551+0000 mon.a [INF] from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]': finished 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.638598+0000 mon.a [INF] from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.638640+0000 mon.a [INF] from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.638690+0000 mon.a [INF] from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.643601+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi178-80695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.644248+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.646044+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.646826+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi178-80663-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:58.708387+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:59.397130+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:59.397391+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:44:59.647 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:59.400589+0000 mon.a [INF] from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:44:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: onexx 2024-02-14T17:44:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:44:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:44:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]': finished 2024-02-14T17:44:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:44:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:44:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi178-80695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:44:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:44:59.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi178-80663-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:44:59.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:44:59.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:44:59.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: osdmap e84: 8 total, 8 up, 8 in 2024-02-14T17:44:59.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:44:59.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:44:59.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:44:59.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi178-80695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:44:59.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:44:59.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:44:59.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:44:59 smithi184 ceph-mon[33536]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi178-80663-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:45:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: onexx 2024-02-14T17:45:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]': finished 2024-02-14T17:45:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi178-80695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi178-80663-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: osdmap e84: 8 total, 8 up, 8 in 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi178-80695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[39811]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi178-80663-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: onexx 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi178-81815-1"}]': finished 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi178-80663-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi178-80695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2951242841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4012409135' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi178-80663-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: osdmap e84: 8 total, 8 up, 8 in 2024-02-14T17:45:00.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:00.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:00.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:00.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi178-80695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:00.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:45:00.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:00.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:44:59 smithi178 ceph-mon[30321]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi178-80663-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:45:01.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:45:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: 14.7 deep-scrub starts 2024-02-14T17:45:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: 14.7 deep-scrub ok 2024-02-14T17:45:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: 14.6 deep-scrub starts 2024-02-14T17:45:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: 14.6 deep-scrub ok 2024-02-14T17:45:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: 14.5 deep-scrub starts 2024-02-14T17:45:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: 14.5 deep-scrub ok 2024-02-14T17:45:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: pgmap v83: 788 pgs: 20 creating+activating, 57 creating+peering, 365 active+clean, 346 unknown; 121 MiB data, 445 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 29 MiB/s wr, 534 op/s 2024-02-14T17:45:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]': finished 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi178-80663-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-4", "overlaypool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: osdmap e85: 8 total, 8 up, 8 in 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-4", "overlaypool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:45:01.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi178-80783-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:01.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:01.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi178-80783-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:01.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: 14.7 deep-scrub starts 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: 14.7 deep-scrub ok 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: 14.6 deep-scrub starts 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: 14.6 deep-scrub ok 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: 14.5 deep-scrub starts 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: 14.5 deep-scrub ok 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: pgmap v83: 788 pgs: 20 creating+activating, 57 creating+peering, 365 active+clean, 346 unknown; 121 MiB data, 445 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 29 MiB/s wr, 534 op/s 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]': finished 2024-02-14T17:45:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi178-80663-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-14T17:45:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-4", "overlaypool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: osdmap e85: 8 total, 8 up, 8 in 2024-02-14T17:45:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-4", "overlaypool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:45:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi178-80783-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi178-80783-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: 14.7 deep-scrub starts 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: 14.7 deep-scrub ok 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: 14.6 deep-scrub starts 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: 14.6 deep-scrub ok 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: 14.5 deep-scrub starts 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: 14.5 deep-scrub ok 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: pgmap v83: 788 pgs: 20 creating+activating, 57 creating+peering, 365 active+clean, 346 unknown; 121 MiB data, 445 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 29 MiB/s wr, 534 op/s 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:45:01.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]': finished 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi178-80663-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-4", "overlaypool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: osdmap e85: 8 total, 8 up, 8 in 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-4", "overlaypool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.24913 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi178-81815-1"}]: dispatch 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi178-80783-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:01.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi178-80783-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:01.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:02.047 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:59.400864+0000 mon.a [INF] from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi178-80695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "e api_io_pp: Running main() from gmock_main.cc 2024-02-14T17:45:02.047 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-02-14T17:45:02.047 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [----------] Global test environment set-up. 2024-02-14T17:45:02.047 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-02-14T17:45:02.047 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: seed 80695 2024-02-14T17:45:02.047 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-02-14T17:45:02.047 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-14T17:45:02.047 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (1183 ms) 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (154 ms) 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (2 ms) 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (2 ms) 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (11 ms) 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (4 ms) 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (5 ms) 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-14T17:45:02.048 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (8 ms) 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (8 ms) 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (2 ms) 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (3 ms) 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (4 ms) 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (4 ms) 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (4 ms) 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (11 ms) 2024-02-14T17:45:02.049 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (5 ms) 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (5 ms) 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (2 ms) 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (6 ms) 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (1423 ms total) 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1133 ms) 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (14 ms) 2024-02-14T17:45:02.050 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-14T17:45:02.051 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (4 ms) 2024-02-14T17:45:02.051 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-14T17:45:02.051 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (4 ms) 2024-02-14T17:45:02.051 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-14T17:45:02.051 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (3 ms) 2024-02-14T17:45:02.187 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN cls: Running main() from gmock_main.cc 2024-02-14T17:45:02.187 INFO:tasks.workunit.client.0.smithi178.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-02-14T17:45:02.187 INFO:tasks.workunit.client.0.smithi178.stdout: cls: [----------] Global test environment set-up. 2024-02-14T17:45:02.187 INFO:tasks.workunit.client.0.smithi178.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-02-14T17:45:02.187 INFO:tasks.workunit.client.0.smithi178.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: cls: [ OK ] NeoRadosCls.DNE (11932 ms) 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (5854 ms) 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: cls: [----------] 2 tests from NeoRadosCls (17786 ms total) 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: cls: 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: cls: [----------] Global test environment tear-down 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: cls: [==========] 2 tests from 1 test suite ran. (17786 ms total) 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: cls: [ PASSED ] 2 tests. 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_1_[81510]: starting. 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_1_[81510]: creating pool ceph_test_rados_open_pools_parallel.smithi178-81428 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_1_[81510]: created object 0... 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_1_[81510]: created object 25... 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_1_[81510]: created object 49... 2024-02-14T17:45:02.188 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_1_[81510]: finishing. 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_1_[81510]: shutting down. 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_2_[81512]: starting. 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_2_[81512]: rados_pool_create. 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_2_[81512]: rados_ioctx_create. 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_2_[81512]: shutting down. 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: ******************************* 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_3_[83034]: starting. 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_3_[83034]: creating pool ceph_test_rados_open_pools_parallel.smithi178-81428 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_3_[83034]: created object 0... 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_3_[83034]: created object 25... 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_3_[83034]: created object 49... 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_3_[83034]: finishing. 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_3_[83034]: shutting down. 2024-02-14T17:45:02.189 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: ******************************* 2024-02-14T17:45:02.190 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_4_[83035]: starting. 2024-02-14T17:45:02.190 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_4_[83035]: rados_pool_create. 2024-02-14T17:45:02.190 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_4_[83035]: rados_ioctx_create. 2024-02-14T17:45:02.190 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: process_4_[83035]: shutting down. 2024-02-14T17:45:02.190 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: ******************************* 2024-02-14T17:45:02.190 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: ******************************* 2024-02-14T17:45:02.190 INFO:tasks.workunit.client.0.smithi178.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-02-14T17:45:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: twoxx 2024-02-14T17:45:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:45:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi178-80695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi178-80695-22"}]': finished 2024-02-14T17:45:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-4", "overlaypool": "test-rados-api-smithi178-80903-4-cache"}]': finished 2024-02-14T17:45:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi178-81815-1"}]': finished 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi178-80783-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: osdmap e86: 8 total, 8 up, 8 in 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/984946609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi178-80647-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.35309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi178-80647-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi178-81099-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi178-80783-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi178-80783-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:02.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi178-80684-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2709307343' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi178-81099-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi178-80684-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi178-81815-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:02.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi178-81815-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: twoxx 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi178-80695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi178-80695-22"}]': finished 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-4", "overlaypool": "test-rados-api-smithi178-80903-4-cache"}]': finished 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi178-81815-1"}]': finished 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi178-80783-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: osdmap e86: 8 total, 8 up, 8 in 2024-02-14T17:45:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:45:02.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/984946609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi178-80647-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:02.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:45:02.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.35309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi178-80647-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:02.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi178-81099-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: twoxx 2024-02-14T17:45:02.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:02.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi178-80695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi178-80695-22"}]': finished 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-4", "overlaypool": "test-rados-api-smithi178-80903-4-cache"}]': finished 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.24913 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi178-81815-1"}]': finished 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi178-80783-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: osdmap e86: 8 total, 8 up, 8 in 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/984946609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi178-80647-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.35309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi178-80647-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:02.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi178-81099-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi178-80783-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi178-80783-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi178-80684-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2709307343' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi178-81099-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi178-80684-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi178-81815-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:02.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi178-81815-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi178-80783-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi178-80783-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi178-80684-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2709307343' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi178-81099-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi178-80684-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:02.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi178-81815-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:02.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi178-81815-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:02.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-14T17:45:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-4-cache", "mode": "writeback"}]': finished 2024-02-14T17:45:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.35309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi178-80647-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:45:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi178-81099-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi178-80684-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi178-81815-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]: dispatch 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:03.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: osdmap e87: 8 total, 8 up, 8 in 2024-02-14T17:45:03.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:03.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:03.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:03.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:03.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:03.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:03.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:03.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]: dispatch 2024-02-14T17:45:03.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:02 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-14T17:45:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-4-cache", "mode": "writeback"}]': finished 2024-02-14T17:45:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.35309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi178-80647-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:45:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi178-81099-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi178-80684-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi178-81815-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:03.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:03.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:03.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:03.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-14T17:45:03.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:03.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-4-cache", "mode": "writeback"}]': finished 2024-02-14T17:45:03.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.35309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi178-80647-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi178-81099-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi178-80684-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi178-81815-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:03.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: osdmap e87: 8 total, 8 up, 8 in 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]: dispatch 2024-02-14T17:45:03.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: osdmap e87: 8 total, 8 up, 8 in 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:03.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:03.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]: dispatch 2024-02-14T17:45:03.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:02 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:03.883 INFO:tasks.workunit.client.0.smithi178.stdout:rasure_code_profile":"testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:03.883 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:59.401969+0000 mon.a [INF] from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:45:03.883 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:59.402163+0000 mon.a [INF] from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:03.883 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:44:59.402708+0000 mon.a [INF] from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi178-80663-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.876161+0000 mon.a [INF] from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.883404+0000 mon.a [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.887144+0000 mon.a [INF] from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.887209+0000 mon.a [INF] from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-4-cache", "mode": "writeback"}]': finished 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.887255+0000 mon.a [INF] from='client.35309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi178-80647-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.887312+0000 mon.a [INF] from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.887365+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.887413+0000 mon.a [INF] from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi178-81099-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.887462+0000 mon.a [INF] from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.887512+0000 mon.a [INF] from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi178-80684-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.887552+0000 mon.a [INF] from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.887592+0000 mon.a [INF] from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi178-81815-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.892346+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/406389666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:03.884 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.892526+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.892991+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.893546+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.893966+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.894369+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:01.896739+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.032740+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.033408+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.185547+0000 mon.a [INF] from='client.34751 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.186225+0000 mon.a [INF] from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.186865+0000 mon.a [INF] from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.187377+0000 mon.a [INF] from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:03.885 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.187877+0000 mon.a [INF] from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:04.146 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:4 cmd: Running main() from gmock_main.cc 2024-02-14T17:45:04.146 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-02-14T17:45:04.146 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [----------] Global test environment set-up. 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (8539 ms) 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (3428 ms) 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (7833 ms) 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [----------] 3 tests from NeoRadosCmd (19800 ms total) 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [----------] Global test environment tear-down 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [==========] 3 tests from 1 test suite ran. (19800 ms total) 2024-02-14T17:45:04.147 INFO:tasks.workunit.client.0.smithi178.stdout: cmd: [ PASSED ] 3 tests. 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_1_[81673]: starting. 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_1_[81673]: creating pool ceph_test_rados_delete_pools_parallel.smithi178-81523 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_1_[81673]: created object 0... 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_1_[81673]: created object 25... 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_1_[81673]: created object 49... 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_1_[81673]: finishing. 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_1_[81673]: shutting down. 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_2_[81675]: starting. 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_2_[81675]: deleting pool ceph_test_rados_delete_pools_parallel.smithi178-81523 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_2_[81675]: shutting down. 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: ******************************* 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_3_[83031]: starting. 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_3_[83031]: creating pool ceph_test_rados_delete_pools_parallel.smithi178-81523 2024-02-14T17:45:04.151 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_3_[83031]: created object 0... 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_3_[83031]: created object 25... 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_3_[83031]: created object 49... 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_3_[83031]: finishing. 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_3_[83031]: shutting down. 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: ******************************* 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_5_[83033]: starting. 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_5_[83033]: listing objects. 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_5_[83033]: listed object 0... 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_5_[83033]: listed object 25... 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_5_[83033]: saw 50 objects 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_5_[83033]: shutting down. 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: ******************************* 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_4_[83032]: starting. 2024-02-14T17:45:04.152 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_4_[83032]: deleting pool ceph_test_rados_delete_pools_parallel.smithi178-81523 2024-02-14T17:45:04.153 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: process_4_[83032]: shutting down. 2024-02-14T17:45:04.153 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: ******************************* 2024-02-14T17:45:04.153 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: ******************************* 2024-02-14T17:45:04.153 INFO:tasks.workunit.client.0.smithi178.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-02-14T17:45:04.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:04 smithi184 ceph-mon[33536]: pgmap v86: 948 pgs: 320 creating+peering, 492 active+clean, 136 unknown; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s wr, 0 op/s 2024-02-14T17:45:04.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:04 smithi184 ceph-mon[33536]: 14.9 deep-scrub starts 2024-02-14T17:45:04.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:04 smithi184 ceph-mon[33536]: 14.9 deep-scrub ok 2024-02-14T17:45:04.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:04.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:04 smithi184 ceph-mon[33536]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi178-80783-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi178-80783-10"}]': finished 2024-02-14T17:45:04.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:04 smithi178 ceph-mon[39811]: pgmap v86: 948 pgs: 320 creating+peering, 492 active+clean, 136 unknown; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s wr, 0 op/s 2024-02-14T17:45:04.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:04 smithi178 ceph-mon[39811]: 14.9 deep-scrub starts 2024-02-14T17:45:04.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:04 smithi178 ceph-mon[39811]: 14.9 deep-scrub ok 2024-02-14T17:45:04.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:04.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:04 smithi178 ceph-mon[39811]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi178-80783-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi178-80783-10"}]': finished 2024-02-14T17:45:04.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:04 smithi178 ceph-mon[30321]: pgmap v86: 948 pgs: 320 creating+peering, 492 active+clean, 136 unknown; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 7.3 MiB/s wr, 0 op/s 2024-02-14T17:45:04.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:04 smithi178 ceph-mon[30321]: 14.9 deep-scrub starts 2024-02-14T17:45:04.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:04 smithi178 ceph-mon[30321]: 14.9 deep-scrub ok 2024-02-14T17:45:04.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:04.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:04 smithi178 ceph-mon[30321]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi178-80783-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi178-80783-10"}]': finished 2024-02-14T17:45:05.171 INFO:tasks.workunit.client.0.smithi178.stdout:5:02.188396+0000 mon.a [INF] from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:05.171 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.189120+0000 mon.a [INF] from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:05.171 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.192803+0000 mon.a [INF] from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]: dispatch 2024-02-14T17:45:05.171 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.193152+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.171 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.918225+0000 mon.a [INF] from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi178-80783-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi178-80783-10"}]': finished 2024-02-14T17:45:05.171 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.918315+0000 mon.a [INF] from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.918382+0000 mon.a [INF] from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.918444+0000 mon.a [INF] from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]': finished 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.918503+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.922588+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.925926+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.932703+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2"}]: dispatch 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:02.934113+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:03.882493+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.142905+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.143248+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/2021810761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.152435+0000 mon.a [INF] from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.152621+0000 mon.a [INF] from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:05.172 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.161383+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.163727+0000 mon.a [INF] from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]': finished 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.163974+0000 mon.a [INF] from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]': finished 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.164121+0000 mon.a [INF] from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]': finished 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.164225+0000 mon.a [INF] from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]': finished 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.164478+0000 mon.a [INF] from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]': finished 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.422466+0000 mon.a [WRN] Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.422801+0000 mon.a [WRN] pool 'PoolQuotaPP_smithi178-80663-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.425370+0000 mon.a [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.425400+0000 mon.a [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.429919+0000 mon.a [INF] from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.429990+0000 mon.a [INF] from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]': finished 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.430050+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:05.173 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.436993+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:45:05.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:45:05.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]': finished 2024-02-14T17:45:05.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:05.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:05.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:05.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2"}]: dispatch 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: osdmap e88: 8 total, 8 up, 8 in 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2021810761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]': finished 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]': finished 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]': finished 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]': finished 2024-02-14T17:45:05.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]': finished 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: osdmap e89: 8 total, 8 up, 8 in 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: pool 'PoolQuotaPP_smithi178-80663-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]': finished 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: osdmap e90: 8 total, 8 up, 8 in 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:05.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.35366 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.473 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]': finished 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2"}]: dispatch 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: osdmap e88: 8 total, 8 up, 8 in 2024-02-14T17:45:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:05.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2021810761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:05.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:05.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]': finished 2024-02-14T17:45:05.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]': finished 2024-02-14T17:45:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.34751 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi178-81146-1"}]': finished 2024-02-14T17:45:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:45:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-4"}]': finished 2024-02-14T17:45:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2918141109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3069035463' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2"}]: dispatch 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: osdmap e88: 8 total, 8 up, 8 in 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2021810761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]: dispatch 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]: dispatch 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi178-80802-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi178-80802-10"}]': finished 2024-02-14T17:45:05.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi178-81099-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi178-81099-7"}]': finished 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]': finished 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]': finished 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]': finished 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: osdmap e89: 8 total, 8 up, 8 in 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: pool 'PoolQuotaPP_smithi178-80663-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]': finished 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: osdmap e90: 8 total, 8 up, 8 in 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.35366 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi178-81091-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi178-81091-7"}]': finished 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi178-80684-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi178-80684-16"}]': finished 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi178-81815-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi178-81815-2"}]': finished 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: osdmap e89: 8 total, 8 up, 8 in 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:05.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: pool 'PoolQuotaPP_smithi178-80663-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi178-81859-1"}]': finished 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-4", "tierpool": "test-rados-api-smithi178-80903-4-cache"}]': finished 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi178-80895-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: osdmap e90: 8 total, 8 up, 8 in 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.35366 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:05.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:06.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:45:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:45:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:45:06.233 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.437828+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-14T17:45:06.233 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.438510+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:06.233 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.478632+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:06.233 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.696491+0000 mon.a [INF] from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:06.233 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:04.698198+0000 mon.a [INF] from='client.35366 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.233 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:03.882654+0000 client.admin [INF] threexx 2024-02-14T17:45:06.233 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.167563+0000 mon.a [INF] from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.233 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.173009+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:06.233 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.412826+0000 mon.a [INF] from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:06.234 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.413287+0000 mon.a [INF] from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:45:06.234 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.658969+0000 mon.a [INF] from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]': finished 2024-02-14T17:45:06.234 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.659048+0000 mon.a [INF] from='client.35366 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:06.234 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.659121+0000 mon.a [INF] from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:06.234 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.659189+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T17:45:06.234 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.659262+0000 mon.a [INF] from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:06.234 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.663303+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:06.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: pgmap v89: 748 pgs: 85 creating+peering, 463 active+clean, 200 unknown; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.3 MiB/s wr, 4 op/s 2024-02-14T17:45:06.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: threexx 2024-02-14T17:45:06.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:06.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:06.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:45:06.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]': finished 2024-02-14T17:45:06.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35366 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:06.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:06.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T17:45:06.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:06.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:06.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi178-81859-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:06.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: osdmap e91: 8 total, 8 up, 8 in 2024-02-14T17:45:06.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/348537437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi178-80647-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:06.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:06.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4035758841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: pgmap v89: 748 pgs: 85 creating+peering, 463 active+clean, 200 unknown; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.3 MiB/s wr, 4 op/s 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: threexx 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]': finished 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35366 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi178-81859-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: osdmap e91: 8 total, 8 up, 8 in 2024-02-14T17:45:06.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/348537437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi178-80647-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4035758841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi178-81859-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-81146-6", "pg_num": 4}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi178-81859-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:06.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-81146-6", "pg_num": 4}]: dispatch 2024-02-14T17:45:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: pgmap v89: 748 pgs: 85 creating+peering, 463 active+clean, 200 unknown; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.3 MiB/s wr, 4 op/s 2024-02-14T17:45:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: threexx 2024-02-14T17:45:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:45:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi178-80695-22"}]': finished 2024-02-14T17:45:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35366 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi178-81859-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/974515126' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi178-81859-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: osdmap e91: 8 total, 8 up, 8 in 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/348537437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi178-80647-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4035758841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi178-81859-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:06.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-81146-6", "pg_num": 4}]: dispatch 2024-02-14T17:45:06.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.663587+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/2990023194' entity='client.adm ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-14T17:45:06.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (5 ms) 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (4 ms) 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (3 ms) 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (4 ms) 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (5 ms) 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (4 ms) 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (9 ms) 2024-02-14T17:45:06.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (4 ms) 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (3 ms) 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (4 ms) 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1210 ms total) 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [----------] Global test environment tear-down 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (22682 ms total) 2024-02-14T17:45:06.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-02-14T17:45:07.008 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: Running main() from gmock_main.cc 2024-02-14T17:45:07.008 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-02-14T17:45:07.008 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: [----------] Global test environment set-up. 2024-02-14T17:45:07.008 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-02-14T17:45:07.008 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-02-14T17:45:07.008 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: handle_notify cookie 94435490796480 notify_id 339302416385 notifier_gid 24895 2024-02-14T17:45:07.008 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (11845 ms) 2024-02-14T17:45:07.008 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-02-14T17:45:07.008 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: Trying... 2024-02-14T17:45:07.008 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: handle_notify cookie 94435503492224 notify_id 369367187456 notifier_gid 25273 2024-02-14T17:45:07.009 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: Waiting for 3.000000000s 2024-02-14T17:45:07.009 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: Timed out. 2024-02-14T17:45:07.009 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: Flushing... 2024-02-14T17:45:07.009 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: Flushed... 2024-02-14T17:45:07.009 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (10681 ms) 2024-02-14T17:45:07.009 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (22527 ms total) 2024-02-14T17:45:07.009 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: 2024-02-14T17:45:07.009 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: [----------] Global test environment tear-down 2024-02-14T17:45:07.009 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (22527 ms total) 2024-02-14T17:45:07.009 INFO:tasks.workunit.client.0.smithi178.stdout: watch_notify: [ PASSED ] 2 tests. 2024-02-14T17:45:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: pgmap v92: 788 pgs: 73 creating+peering, 586 active+clean, 129 unknown; 145 MiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-02-14T17:45:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-14T17:45:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:45:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: fourxx 2024-02-14T17:45:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.25405 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi178-80647-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-81146-6", "pg_num": 4}]: dispatch 2024-02-14T17:45:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]': finished 2024-02-14T17:45:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T17:45:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.25405 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi178-80647-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-81146-6", "pg_num": 4}]': finished 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: osdmap e92: 8 total, 8 up, 8 in 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[30321]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: pgmap v92: 788 pgs: 73 creating+peering, 586 active+clean, 129 unknown; 145 MiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: fourxx 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.25405 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi178-80647-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-81146-6", "pg_num": 4}]: dispatch 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]': finished 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:07.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.25405 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi178-80647-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-81146-6", "pg_num": 4}]': finished 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: osdmap e92: 8 total, 8 up, 8 in 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:07.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:07 smithi178 ceph-mon[39811]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: pgmap v92: 788 pgs: 73 creating+peering, 586 active+clean, 129 unknown; 145 MiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-02-14T17:45:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-14T17:45:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3232841733' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:45:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: fourxx 2024-02-14T17:45:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.25405 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi178-80647-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-81146-6", "pg_num": 4}]: dispatch 2024-02-14T17:45:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:07.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.35282 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]': finished 2024-02-14T17:45:07.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T17:45:07.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.25405 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi178-80647-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-81146-6", "pg_num": 4}]': finished 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: osdmap e92: 8 total, 8 up, 8 in 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.25309 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:07.724 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:07 smithi184 ceph-mon[33536]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:07.772 INFO:tasks.workunit.client.0.smithi178.stdout: list: Running main() from gmock_main.cc 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [----------] Global test environment set-up. 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [----------] 3 tests from NeoradosList 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [ OK ] NeoradosList.ListObjects (11893 ms) 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [ OK ] NeoradosList.ListObjectsNS (5853 ms) 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [ OK ] NeoradosList.ListObjectsMany (5586 ms) 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [----------] 3 tests from NeoradosList (23332 ms total) 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [----------] Global test environment tear-down 2024-02-14T17:45:07.773 INFO:tasks.workunit.client.0.smithi178.stdout: list: [==========] 3 tests from 1 test suite ran. (23332 ms total) 2024-02-14T17:45:07.774 INFO:tasks.workunit.client.0.smithi178.stdout: list: [ PASSED ] 3 tests. 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi178-81859-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi178-81859-2"}]': finished 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.35330 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1"}]': finished 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: osdmap e93: 8 total, 8 up, 8 in 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "overlaypool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "overlaypool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi178-81192-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:09.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi178-81192-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3846124585' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.35435 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:09.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:08 smithi184 ceph-mon[33536]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:09.273 INFO:tasks.workunit.client.0.smithi178.stdout:in' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi178-81859-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.666346+0000 mon.b [INF] from='client.? v1:172.21.15.178:0/348537437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi178-80647-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.674990+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3241377214' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.675954+0000 mon.a [INF] from='client.35282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi178-80695-22"}]: dispatch 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.676384+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/4035758841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.676545+0000 mon.a [INF] from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi178-81859-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.678182+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.678494+0000 mon.a [INF] from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: got: 2024-02-14T17:45:05.783881+0000 mon.c [INF] from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-81146-6", "pg_num": 4}]: dispatch 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (12942 ms) 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (25020 ms total) 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: 2024-02-14T17:45:09.274 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [----------] Global test environment tear-down 2024-02-14T17:45:09.275 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (25020 ms total) 2024-02-14T17:45:09.275 INFO:tasks.workunit.client.0.smithi178.stdout: api_cmd: [ PASSED ] 4 tests. 2024-02-14T17:45:09.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-14T17:45:09.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:09.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi178-81859-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi178-81859-2"}]': finished 2024-02-14T17:45:09.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:09.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.35330 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1"}]': finished 2024-02-14T17:45:09.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:09.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: osdmap e93: 8 total, 8 up, 8 in 2024-02-14T17:45:09.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "overlaypool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:09.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:09.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "overlaypool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:09.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi178-81192-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:09.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi178-81192-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:09.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:09.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:09.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:09.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:09.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:09.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3846124585' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:09.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.35435 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:09.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:09.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:09.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:09.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[30321]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:09.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.25309 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi178-81859-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi178-81859-2"}]': finished 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.35330 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi178-80895-1","app":"app1","key":"key1"}]': finished 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: osdmap e93: 8 total, 8 up, 8 in 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "overlaypool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "overlaypool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi178-81192-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi178-81192-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:09.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:09.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:09.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:09.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3846124585' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:09.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.35435 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:09.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:09.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:09.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:09.339 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:08 smithi178 ceph-mon[39811]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "overlaypool": "test-rados-api-smithi178-81146-6"}]': finished 2024-02-14T17:45:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]': finished 2024-02-14T17:45:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.35435 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi178-81815-2"}]': finished 2024-02-14T17:45:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]': finished 2024-02-14T17:45:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]': finished 2024-02-14T17:45:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]': finished 2024-02-14T17:45:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81146-6", "mode": "writeback"}]: dispatch 2024-02-14T17:45:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: osdmap e94: 8 total, 8 up, 8 in 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/700101697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi178-80647-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi178-81192-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]': finished 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[30321]: osdmap e95: 8 total, 8 up, 8 in 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "overlaypool": "test-rados-api-smithi178-81146-6"}]': finished 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]': finished 2024-02-14T17:45:10.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.35435 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi178-81815-2"}]': finished 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]': finished 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]': finished 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]': finished 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81146-6", "mode": "writeback"}]: dispatch 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: osdmap e94: 8 total, 8 up, 8 in 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/700101697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi178-80647-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi178-81192-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]': finished 2024-02-14T17:45:10.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:10 smithi178 ceph-mon[39811]: osdmap e95: 8 total, 8 up, 8 in 2024-02-14T17:45:10.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "overlaypool": "test-rados-api-smithi178-81146-6"}]': finished 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi178-80783-10"}]': finished 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.35435 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi178-81815-2"}]': finished 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi178-81091-7"}]': finished 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi178-81099-7"}]': finished 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi178-80684-16"}]': finished 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1874826680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81146-6", "mode": "writeback"}]: dispatch 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3058252982' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4149682109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2794804310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1061826483' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:10.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: osdmap e94: 8 total, 8 up, 8 in 2024-02-14T17:45:10.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/700101697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi178-80647-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:10.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:10.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi178-81192-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]': finished 2024-02-14T17:45:10.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-mon[33536]: osdmap e95: 8 total, 8 up, 8 in 2024-02-14T17:45:10.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-02-14T17:45:10.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-02-14T17:45:10.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: seed 81091 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (309 ms) 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (22 ms) 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (5 ms) 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (13 ms) 2024-02-14T17:45:10.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (349 ms total) 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1606 ms) 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1 ms) 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (2 ms) 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (11 ms) 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1620 ms total) 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (26506 ms total) 2024-02-14T17:45:10.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-02-14T17:45:10.727 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: Running main() from gmock_main.cc 2024-02-14T17:45:10.727 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-02-14T17:45:10.727 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: [----------] Global test environment set-up. 2024-02-14T17:45:10.727 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-02-14T17:45:10.727 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-02-14T17:45:10.727 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (16460 ms) 2024-02-14T17:45:10.727 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-02-14T17:45:10.727 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (9826 ms) 2024-02-14T17:45:10.728 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (26286 ms total) 2024-02-14T17:45:10.728 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: 2024-02-14T17:45:10.728 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: [----------] Global test environment tear-down 2024-02-14T17:45:10.728 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (26286 ms total) 2024-02-14T17:45:10.728 INFO:tasks.workunit.client.0.smithi178.stdout: ec_io: [ PASSED ] 2 tests. 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: Running main() from gmock_main.cc 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [----------] Global test environment set-up. 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (335 ms) 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: no timeout :/ 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: no timeout :/ 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: no timeout :/ 2024-02-14T17:45:10.739 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: no timeout :/ 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: no timeout :/ 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (43 ms) 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (8 ms) 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.Checksum (3 ms) 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (6 ms) 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (3 ms) 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (5 ms) 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-02-14T17:45:10.740 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.TruncTest (3 ms) 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (4 ms) 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (3 ms) 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.RmXattr (11 ms) 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIo.XattrIter (4 ms) 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [----------] 14 tests from LibRadosIo (430 ms total) 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1605 ms) 2024-02-14T17:45:10.741 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (9 ms) 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (7 ms) 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (7 ms) 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (3 ms) 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (4 ms) 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-02-14T17:45:10.742 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (5 ms) 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1659 ms total) 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [----------] Global test environment tear-down 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [==========] 24 tests from 2 test suites ran. (26677 ms total) 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_io: [ PASSED ] 24 tests. 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: Running main() from gmock_main.cc 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [----------] Global test environment set-up. 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (256 ms) 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-02-14T17:45:10.743 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLock.LockShared (84 ms) 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1032 ms) 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1008 ms) 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (7 ms) 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLock.Unlock (150 ms) 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (5 ms) 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [----------] 8 tests from LibRadosLock (2545 ms total) 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-02-14T17:45:10.744 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1583 ms) 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (2 ms) 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1005 ms) 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1005 ms) 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (3 ms) 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (3 ms) 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (5 ms) 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (2 ms) 2024-02-14T17:45:10.745 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3608 ms total) 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [----------] Global test environment tear-down 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (26632 ms total) 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock: [ PASSED ] 16 tests. 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: Running main() from gmock_main.cc 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [----------] Global test environment set-up. 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ OK ] LibRadosStat.Stat (333 ms) 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (11 ms) 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ OK ] LibRadosStat.StatNS (13 ms) 2024-02-14T17:45:10.746 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1 ms) 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (1 ms) 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [----------] 5 tests from LibRadosStat (359 ms total) 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1582 ms) 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (31 ms) 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (3 ms) 2024-02-14T17:45:10.747 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1616 ms total) 2024-02-14T17:45:10.748 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: 2024-02-14T17:45:10.748 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [----------] Global test environment tear-down 2024-02-14T17:45:10.748 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (26518 ms total) 2024-02-14T17:45:10.748 INFO:tasks.workunit.client.0.smithi178.stdout: api_stat: [ PASSED ] 9 tests. 2024-02-14T17:45:11.405 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:45:10 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:45:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: pgmap v95: 792 pgs: 77 creating+peering, 455 active+clean, 260 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-02-14T17:45:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: pgmap v98: 664 pgs: 40 creating+peering, 388 active+clean, 236 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81146-6", "mode": "writeback"}]: dispatch 2024-02-14T17:45:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25450 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi178-80647-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]': finished 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81146-6", "mode": "writeback"}]': finished 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]': finished 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]': finished 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]': finished 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi178-81815-2"}]': finished 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25450 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi178-80647-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: osdmap e96: 8 total, 8 up, 8 in 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2788220580' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25468 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:11.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:11.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:11.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:11.851 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: pgmap v95: 792 pgs: 77 creating+peering, 455 active+clean, 260 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-02-14T17:45:11.852 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: pgmap v98: 664 pgs: 40 creating+peering, 388 active+clean, 236 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:11.852 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:11.852 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:11.852 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81146-6", "mode": "writeback"}]: dispatch 2024-02-14T17:45:11.852 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:11.852 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:11.852 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:11.852 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:11.852 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25450 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi178-80647-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:11.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T17:45:11.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]': finished 2024-02-14T17:45:11.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81146-6", "mode": "writeback"}]': finished 2024-02-14T17:45:11.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]': finished 2024-02-14T17:45:11.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]': finished 2024-02-14T17:45:11.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]': finished 2024-02-14T17:45:11.853 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi178-81815-2"}]': finished 2024-02-14T17:45:11.854 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25450 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi178-80647-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:11.854 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: osdmap e96: 8 total, 8 up, 8 in 2024-02-14T17:45:11.854 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2788220580' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.854 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:11.854 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25468 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.854 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:11.854 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.854 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.854 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:11.855 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:11.855 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:11.855 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: pgmap v95: 792 pgs: 77 creating+peering, 455 active+clean, 260 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.7 KiB/s wr, 3 op/s 2024-02-14T17:45:11.855 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: pgmap v98: 664 pgs: 40 creating+peering, 388 active+clean, 236 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:11.855 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:11.855 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]: dispatch 2024-02-14T17:45:11.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81146-6", "mode": "writeback"}]: dispatch 2024-02-14T17:45:11.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]: dispatch 2024-02-14T17:45:11.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]: dispatch 2024-02-14T17:45:11.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25294 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]: dispatch 2024-02-14T17:45:11.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi178-81815-2"}]: dispatch 2024-02-14T17:45:11.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25450 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi178-80647-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:11.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T17:45:11.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi178-80783-10"}]': finished 2024-02-14T17:45:11.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81146-6", "mode": "writeback"}]': finished 2024-02-14T17:45:11.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi178-80684-16"}]': finished 2024-02-14T17:45:11.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi178-81099-7"}]': finished 2024-02-14T17:45:11.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25294 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi178-81091-7"}]': finished 2024-02-14T17:45:11.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi178-81815-2"}]': finished 2024-02-14T17:45:11.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25450 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi178-80647-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:11.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: osdmap e96: 8 total, 8 up, 8 in 2024-02-14T17:45:11.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2788220580' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:11.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25468 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.858 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:11.858 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.858 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:11.858 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:11.858 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:11.858 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[30321]: from='client.25468 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[30321]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]': finished 2024-02-14T17:45:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[30321]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi178-81859-2"}]': finished 2024-02-14T17:45:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3005857725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[30321]: osdmap e97: 8 total, 8 up, 8 in 2024-02-14T17:45:12.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:12.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:12.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[39811]: from='client.25468 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:12.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[39811]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]': finished 2024-02-14T17:45:12.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:12.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[39811]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi178-81859-2"}]': finished 2024-02-14T17:45:12.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:12.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:12.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3005857725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:12.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:12 smithi178 ceph-mon[39811]: osdmap e97: 8 total, 8 up, 8 in 2024-02-14T17:45:12.947 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-02-14T17:45:12.947 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-02-14T17:45:12.947 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-02-14T17:45:12.947 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-02-14T17:45:12.947 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: seed 80802 2024-02-14T17:45:12.947 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-02-14T17:45:12.947 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (272 ms) 2024-02-14T17:45:12.947 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-02-14T17:45:12.947 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (14 ms) 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1007 ms) 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1009 ms) 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (4 ms) 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (4 ms) 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (4 ms) 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (16 ms) 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2331 ms total) 2024-02-14T17:45:12.948 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1604 ms) 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (7 ms) 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1065 ms) 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1119 ms) 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (13 ms) 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (7 ms) 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-02-14T17:45:12.949 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (6 ms) 2024-02-14T17:45:12.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-02-14T17:45:12.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (7 ms) 2024-02-14T17:45:12.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3828 ms total) 2024-02-14T17:45:12.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: 2024-02-14T17:45:12.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-02-14T17:45:12.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (28841 ms total) 2024-02-14T17:45:12.950 INFO:tasks.workunit.client.0.smithi178.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: Running main() from gmock_main.cc 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [----------] Global test environment set-up. 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ OK ] NeoRadosPools.PoolList (10479 ms) 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (3716 ms) 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (3533 ms) 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (5551 ms) 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (2993 ms) 2024-02-14T17:45:12.957 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-02-14T17:45:12.958 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2218 ms) 2024-02-14T17:45:12.958 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [----------] 6 tests from NeoRadosPools (28490 ms total) 2024-02-14T17:45:12.958 INFO:tasks.workunit.client.0.smithi178.stdout: pool: 2024-02-14T17:45:12.958 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [----------] Global test environment tear-down 2024-02-14T17:45:12.958 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [==========] 6 tests from 1 test suite ran. (28490 ms total) 2024-02-14T17:45:12.958 INFO:tasks.workunit.client.0.smithi178.stdout: pool: [ PASSED ] 6 tests. 2024-02-14T17:45:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:12 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:12 smithi184 ceph-mon[33536]: from='client.25468 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:12 smithi184 ceph-mon[33536]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi178-80802-10"}]': finished 2024-02-14T17:45:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:12 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:12 smithi184 ceph-mon[33536]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi178-81859-2"}]': finished 2024-02-14T17:45:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2344917393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2990023194' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3005857725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:12 smithi184 ceph-mon[33536]: osdmap e97: 8 total, 8 up, 8 in 2024-02-14T17:45:13.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:45:13.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:45:13.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [ OK ] LibRadosService.StatusFormat (4191 ms) 2024-02-14T17:45:13.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [ RUN ] LibRadosService.Status 2024-02-14T17:45:13.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [ OK ] LibRadosService.Status (20049 ms) 2024-02-14T17:45:13.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [----------] 4 tests from LibRadosService (29303 ms total) 2024-02-14T17:45:13.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: 2024-02-14T17:45:13.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [----------] Global test environment tear-down 2024-02-14T17:45:13.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [==========] 4 tests from 1 test suite ran. (29303 ms total) 2024-02-14T17:45:13.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_service: [ PASSED ] 4 tests. 2024-02-14T17:45:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:13 smithi184 ceph-mon[33536]: pgmap v101: 592 pgs: 136 creating+peering, 392 active+clean, 64 unknown; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:13 smithi184 ceph-mon[33536]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:13 smithi184 ceph-mon[33536]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:13 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:13 smithi184 ceph-mon[33536]: from='client.28054 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:13 smithi184 ceph-mon[33536]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]': finished 2024-02-14T17:45:14.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[30321]: pgmap v101: 592 pgs: 136 creating+peering, 392 active+clean, 64 unknown; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:14.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[30321]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:14.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[30321]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:14.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:14.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:14.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[30321]: from='client.28054 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:14.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:14.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[30321]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]': finished 2024-02-14T17:45:14.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[39811]: pgmap v101: 592 pgs: 136 creating+peering, 392 active+clean, 64 unknown; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:14.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[39811]: from='client.25261 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]: dispatch 2024-02-14T17:45:14.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[39811]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi178-81859-2"}]: dispatch 2024-02-14T17:45:14.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:14.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:14.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[39811]: from='client.28054 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:14.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:14.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:13 smithi178 ceph-mon[39811]: from='client.25261 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi178-80802-10"}]': finished 2024-02-14T17:45:14.116 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-02-14T17:45:14.116 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-02-14T17:45:14.116 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-02-14T17:45:14.116 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-02-14T17:45:14.116 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-02-14T17:45:14.116 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: notify 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (3510 ms) 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (12 ms) 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (3522 ms total) 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: notify 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (111 ms) 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: notify 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (5122 ms) 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (39 ms) 2024-02-14T17:45:14.117 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (6 ms) 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: handle_notify cookie 94491656692496 notify_id 412316860417 notifier_gid 35372 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (6 ms) 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: handle_notify cookie 94491656692496 notify_id 412316860419 notifier_gid 35372 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (10 ms) 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: handle_notify cookie 94491656692496 notify_id 412316860419 notifier_gid 35372 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (25 ms) 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: handle_notify cookie 94491656692496 notify_id 412316860418 notifier_gid 35372 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (25 ms) 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-02-14T17:45:14.118 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: handle_notify cookie 94491657587904 notify_id 412316860420 notifier_gid 35372 2024-02-14T17:45:14.119 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (19 ms) 2024-02-14T17:45:14.119 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-02-14T17:45:14.119 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: handle_notify cookie 94491657590944 notify_id 412316860420 notifier_gid 35372 2024-02-14T17:45:14.119 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (30 ms) 2024-02-14T17:45:14.119 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-02-14T17:45:14.119 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: trying... 2024-02-14T17:45:14.119 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: handle_notify cookie 94491657587904 notify_id 412316860421 notifier_gid 35372 2024-02-14T17:45:14.119 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: timed out 2024-02-14T17:45:14.119 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: flushing 2024-02-14T17:45:15.717 INFO:tasks.workunit.client.0.smithi178.stdout: a api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-02-14T17:45:15.717 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [----------] Global test environment set-up. 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5065 ms) 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (45 ms) 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (21034 ms) 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: attempt 0 of 20 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: failed to deregister: 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: { 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "epoch": 19, 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "modified": "2024-02-14T17:45:11.352413+0000", 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "services": { 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "laundry": { 2024-02-14T17:45:15.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "daemons": { 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "summary": "", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "close-test-pid81337": { 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "start_epoch": 19, 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "start_stamp": "2024-02-14T17:45:10.607271+0000", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "gid": 37733, 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "addr": "172.21.15.178:0/1533877325", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "metadata": { 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "arch": "x86_64", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_release": "squid", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev)", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_version_short": "19.0.0-1430-g8727ce93", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro": "centos", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro_version": "9", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "foo": "bar", 2024-02-14T17:45:15.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "hostname": "smithi178", 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Fri Feb 9 18:10:28 UTC 2024", 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "kernel_version": "5.14.0-420.el9.x86_64", 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "mem_total_kb": "32488280", 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "os": "Linux", 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "this": "that" 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: }, 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "task_status": {} 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: }, 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "pid81261": { 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "start_epoch": 18, 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "start_stamp": "2024-02-14T17:44:53.556589+0000", 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "gid": 25135, 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "addr": "172.21.15.178:0/204979982", 2024-02-14T17:45:15.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "metadata": { 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "arch": "x86_64", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_release": "squid", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev)", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_version_short": "19.0.0-1430-g8727ce93", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro": "centos", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro_version": "9", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "foo": "bar", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "hostname": "smithi178", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Fri Feb 9 18:10:28 UTC 2024", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "kernel_version": "5.14.0-420.el9.x86_64", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "mem_total_kb": "32488280", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "os": "Linux", 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "this": "that" 2024-02-14T17:45:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: }, 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "task_status": {} 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: }, 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "pid81337": { 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "start_epoch": 19, 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "start_stamp": "2024-02-14T17:45:09.448229+0000", 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "gid": 25051, 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "addr": "172.21.15.178:0/4250131728", 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "metadata": { 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "arch": "x86_64", 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_release": "squid", 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev)", 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_version_short": "19.0.0-1430-g8727ce93", 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro": "centos", 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-14T17:45:15.722 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro_version": "9", 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "foo": "bar", 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "hostname": "smithi178", 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Fri Feb 9 18:10:28 UTC 2024", 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "kernel_version": "5.14.0-420.el9.x86_64", 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "mem_total_kb": "32488280", 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "os": "Linux", 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "this": "that" 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: }, 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "task_status": {} 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: } 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: } 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: }, 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "rgw": { 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "daemons": { 2024-02-14T17:45:15.723 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "summary": "", 2024-02-14T17:45:15.724 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "34367": { 2024-02-14T17:45:15.724 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "start_epoch": 12, 2024-02-14T17:45:15.724 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "start_stamp": "2024-02-14T17:42:05.188512+0000", 2024-02-14T17:45:15.724 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "gid": 34367, 2024-02-14T17:45:15.724 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "addr": "172.21.15.178:0/2582019531", 2024-02-14T17:45:15.724 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "metadata": { 2024-02-14T17:45:15.724 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "arch": "x86_64", 2024-02-14T17:45:15.724 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_release": "squid", 2024-02-14T17:45:15.724 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev)", 2024-02-14T17:45:15.724 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "ceph_version_short": "19.0.0-1430-g8727ce93", 2024-02-14T17:45:15.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi178-81859-2"}]': finished 2024-02-14T17:45:15.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.28054 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:15.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:15.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3561682521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi178-80957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi178-81859-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: osdmap e98: 8 total, 8 up, 8 in 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.28057 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi178-80957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]': finished 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.28057 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi178-80957-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: osdmap e99: 8 total, 8 up, 8 in 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi178-81859-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3305321108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/463101301' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:15.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[30321]: from='client.40109 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.858 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:45:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:45:15.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi178-81859-2"}]': finished 2024-02-14T17:45:15.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.28054 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:15.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:15.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3561682521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi178-80957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi178-81859-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:15.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: osdmap e98: 8 total, 8 up, 8 in 2024-02-14T17:45:15.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.28057 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi178-80957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]': finished 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.28057 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi178-80957-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: osdmap e99: 8 total, 8 up, 8 in 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi178-81859-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3305321108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/463101301' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:15.859 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:15 smithi178 ceph-mon[39811]: from='client.40109 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.865 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_p api_watch_notify: Running main() from gmock_main.cc 2024-02-14T17:45:15.865 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-02-14T17:45:15.865 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-02-14T17:45:15.865 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-02-14T17:45:15.865 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-02-14T17:45:15.865 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify_test_cb 2024-02-14T17:45:15.865 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (313 ms) 2024-02-14T17:45:15.865 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93832701351072 err -107 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1008 ms) 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93832701353120 err -107 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1009 ms) 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_cb from 34757 notify_id 343597383680 cookie 93832701374736 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (6 ms) 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_cb from 34757 notify_id 343597383681 cookie 93832701374736 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (42 ms) 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_cb from 34757 notify_id 343597383681 cookie 93832701374736 2024-02-14T17:45:15.866 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (124 ms) 2024-02-14T17:45:15.867 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-02-14T17:45:15.867 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_cb from 34757 notify_id 347892350976 cookie 93832701374736 2024-02-14T17:45:15.867 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_cb from 34757 notify_id 347892350976 cookie 93832701425040 2024-02-14T17:45:15.867 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (42 ms) 2024-02-14T17:45:15.867 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-02-14T17:45:15.867 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_cb from 34757 notify_id 347892350978 cookie 93832701425040 2024-02-14T17:45:15.867 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_cb from 34757 notify_id 352187318275 cookie 93832701425040 2024-02-14T17:45:15.867 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3005 ms) 2024-02-14T17:45:15.867 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-02-14T17:45:15.867 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93832701374608 err -107 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_cb from 34757 notify_id 377957122051 cookie 93832701374608 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5007 ms) 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93832701425040 err -107 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1057 ms) 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (11613 ms total) 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: watch_notify_test_cb 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1203 ms) 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1203 ms total) 2024-02-14T17:45:15.868 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: 2024-02-14T17:45:15.869 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-02-14T17:45:15.869 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (31641 ms total) 2024-02-14T17:45:15.869 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-02-14T17:45:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi178-81859-2"}]': finished 2024-02-14T17:45:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.28054 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3561682521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi178-80957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi178-81859-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: osdmap e98: 8 total, 8 up, 8 in 2024-02-14T17:45:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.28057 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi178-80957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:15.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:15.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi178-81192-12"}]': finished 2024-02-14T17:45:15.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.28057 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi178-80957-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:15.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: osdmap e99: 8 total, 8 up, 8 in 2024-02-14T17:45:15.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/190703999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:15.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi178-81859-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:15.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3305321108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/463101301' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:15.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:15.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:15 smithi184 ceph-mon[33536]: from='client.40109 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: pgmap v104: 520 pgs: 328 active+clean, 192 unknown; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:45:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.29293 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi178-81859-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.40109 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi178-81859-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2716905472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi178-80647-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: osdmap e100: 8 total, 8 up, 8 in 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi178-81859-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.30313 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi178-80647-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3170530896' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]': finished 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.29293 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.30313 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi178-80647-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:16.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:16.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: osdmap e101: 8 total, 8 up, 8 in 2024-02-14T17:45:16.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:17.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: pgmap v104: 520 pgs: 328 active+clean, 192 unknown; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:45:17.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:17.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:17.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:17.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:17.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.29293 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:17.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi178-81859-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:17.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.40109 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:17.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi178-81859-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:17.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2716905472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi178-80647-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:17.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: osdmap e100: 8 total, 8 up, 8 in 2024-02-14T17:45:17.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi178-81859-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:17.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.30313 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi178-80647-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:17.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3170530896' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:17.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:17.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]': finished 2024-02-14T17:45:17.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:17.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.29293 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:17.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.30313 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi178-80647-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:17.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:17.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: osdmap e101: 8 total, 8 up, 8 in 2024-02-14T17:45:17.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: pgmap v104: 520 pgs: 328 active+clean, 192 unknown; 145 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.25432 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]: dispatch 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.29293 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi178-81859-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.40109 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi178-81859-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2716905472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi178-80647-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: osdmap e100: 8 total, 8 up, 8 in 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi178-81859-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.30313 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi178-80647-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3170530896' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:17.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:17.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.25432 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi178-81192-12"}]': finished 2024-02-14T17:45:17.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:17.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.29293 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:17.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.30313 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi178-80647-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:17.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:17.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: osdmap e101: 8 total, 8 up, 8 in 2024-02-14T17:45:17.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:18.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[30321]: pgmap v107: 648 pgs: 64 creating+peering, 406 active+clean, 178 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:45:18.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.184:0/1011445190' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:18.091 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[30321]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi178-81859-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi178-81859-3"}]': finished 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-7"}]': finished 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[30321]: osdmap e102: 8 total, 8 up, 8 in 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2472377438' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[30321]: from='client.42989 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[39811]: pgmap v107: 648 pgs: 64 creating+peering, 406 active+clean, 178 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.184:0/1011445190' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[39811]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi178-81859-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi178-81859-3"}]': finished 2024-02-14T17:45:18.092 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-7"}]': finished 2024-02-14T17:45:18.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[39811]: osdmap e102: 8 total, 8 up, 8 in 2024-02-14T17:45:18.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2472377438' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:18.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:18.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[39811]: from='client.42989 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:18.093 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:17 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:18.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:17 smithi184 ceph-mon[33536]: pgmap v107: 648 pgs: 64 creating+peering, 406 active+clean, 178 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:45:18.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:17 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.184:0/1011445190' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:18.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:17 smithi184 ceph-mon[33536]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi178-81859-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi178-81859-3"}]': finished 2024-02-14T17:45:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:17 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-7"}]': finished 2024-02-14T17:45:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:17 smithi184 ceph-mon[33536]: osdmap e102: 8 total, 8 up, 8 in 2024-02-14T17:45:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2472377438' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:17 smithi184 ceph-mon[33536]: from='client.42989 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:17 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:19 smithi184 ceph-mon[33536]: from='client.42989 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:45:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:19 smithi184 ceph-mon[33536]: osdmap e103: 8 total, 8 up, 8 in 2024-02-14T17:45:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:19 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[30321]: from='client.42989 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:45:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[30321]: osdmap e103: 8 total, 8 up, 8 in 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[39811]: from='client.42989 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[39811]: osdmap e103: 8 total, 8 up, 8 in 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7"}]: dispatch 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:20.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: pgmap v110: 656 pgs: 90 creating+peering, 430 active+clean, 136 unknown; 145 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-02-14T17:45:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7"}]': finished 2024-02-14T17:45:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/163969511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3001242605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: osdmap e104: 8 total, 8 up, 8 in 2024-02-14T17:45:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: from='client.45224 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: from='client.36046 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: osdmap e105: 8 total, 8 up, 8 in 2024-02-14T17:45:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1430384802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi178-80647-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-mon[33536]: from='client.45230 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi178-80647-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: pgmap v110: 656 pgs: 90 creating+peering, 430 active+clean, 136 unknown; 145 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-02-14T17:45:21.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7"}]': finished 2024-02-14T17:45:21.035 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/163969511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.035 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3001242605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.035 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:21.035 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: osdmap e104: 8 total, 8 up, 8 in 2024-02-14T17:45:21.035 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:21.035 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: from='client.45224 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.035 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: from='client.36046 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.035 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: osdmap e105: 8 total, 8 up, 8 in 2024-02-14T17:45:21.035 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1430384802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi178-80647-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.035 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[39811]: from='client.45230 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi178-80647-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.036 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: pgmap v110: 656 pgs: 90 creating+peering, 430 active+clean, 136 unknown; 145 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-02-14T17:45:21.036 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-7"}]': finished 2024-02-14T17:45:21.036 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/163969511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.036 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3001242605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.036 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:21.037 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: osdmap e104: 8 total, 8 up, 8 in 2024-02-14T17:45:21.037 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:21.037 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: from='client.45224 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.037 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: from='client.36046 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.037 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: osdmap e105: 8 total, 8 up, 8 in 2024-02-14T17:45:21.037 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1430384802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi178-80647-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.037 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:20 smithi178 ceph-mon[30321]: from='client.45230 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi178-80647-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout:p: "container_hostname": "smithi178", 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b", 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro": "centos", 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "distro_version": "8", 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "hostname": "smithi178", 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "id": "foo.a", 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Fri Feb 9 18:10:28 UTC 2024", 2024-02-14T17:45:21.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "kernel_version": "5.14.0-420.el9.x86_64", 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "mem_total_kb": "32488280", 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "num_handles": "1", 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "os": "Linux", 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "pid": "7", 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "realm_id": "", 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "realm_name": "", 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "zone_id": "76a06a30-8223-4955-aca0-3802f0b4de6f", 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "zone_name": "default", 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "zonegroup_id": "b19a8733-4d52-4462-bb0d-327d313b2187", 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "zonegroup_name": "default" 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: }, 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: "task_status": {} 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: } 2024-02-14T17:45:21.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: } 2024-02-14T17:45:21.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: } 2024-02-14T17:45:21.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: } 2024-02-14T17:45:21.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: } 2024-02-14T17:45:21.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: 2024-02-14T17:45:21.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: attempt 1 of 20 2024-02-14T17:45:21.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10626 ms) 2024-02-14T17:45:21.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (36770 ms total) 2024-02-14T17:45:21.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: 2024-02-14T17:45:21.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [----------] Global test environment tear-down 2024-02-14T17:45:21.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (36770 ms total) 2024-02-14T17:45:21.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-02-14T17:45:21.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:45:20 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:45:22.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[30321]: pgmap v112: 560 pgs: 424 active+clean, 136 unknown; 145 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-02-14T17:45:22.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:22.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/30257205' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:22.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[30321]: from='client.45224 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:22.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[30321]: from='client.36046 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:22.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[30321]: from='client.45230 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi178-80647-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:22.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[30321]: osdmap e106: 8 total, 8 up, 8 in 2024-02-14T17:45:22.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3942824759' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:22.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:22.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[39811]: pgmap v112: 560 pgs: 424 active+clean, 136 unknown; 145 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-02-14T17:45:22.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:22.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/30257205' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:22.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[39811]: from='client.45224 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:22.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[39811]: from='client.36046 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:22.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[39811]: from='client.45230 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi178-80647-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:22.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[39811]: osdmap e106: 8 total, 8 up, 8 in 2024-02-14T17:45:22.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3942824759' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:22.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:22.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:22 smithi184 ceph-mon[33536]: pgmap v112: 560 pgs: 424 active+clean, 136 unknown; 145 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-02-14T17:45:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/30257205' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:22 smithi184 ceph-mon[33536]: from='client.45224 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:22 smithi184 ceph-mon[33536]: from='client.36046 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi178-80663-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:22 smithi184 ceph-mon[33536]: from='client.45230 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi178-80647-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:22 smithi184 ceph-mon[33536]: osdmap e106: 8 total, 8 up, 8 in 2024-02-14T17:45:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3942824759' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:23 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:23 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.184:0/1011445190' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:23 smithi184 ceph-mon[33536]: from='client.38119 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:23 smithi184 ceph-mon[33536]: osdmap e107: 8 total, 8 up, 8 in 2024-02-14T17:45:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:23 smithi184 ceph-mon[33536]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4"}]: dispatch 2024-02-14T17:45:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.184:0/1011445190' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[39811]: from='client.38119 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[39811]: osdmap e107: 8 total, 8 up, 8 in 2024-02-14T17:45:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[39811]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4"}]: dispatch 2024-02-14T17:45:23.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:23.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.184:0/1011445190' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:45:23.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:23.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[30321]: from='client.38119 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:23.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:23.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[30321]: osdmap e107: 8 total, 8 up, 8 in 2024-02-14T17:45:23.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[30321]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:23.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:23.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4"}]: dispatch 2024-02-14T17:45:24.487 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: pgmap v116: 552 pgs: 160 creating+peering, 360 active+clean, 32 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:45:24.487 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: pgmap v118: 648 pgs: 160 creating+peering, 360 active+clean, 128 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:45:24.487 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:24.487 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4"}]: dispatch 2024-02-14T17:45:24.487 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: from='client.38119 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:24.487 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi178-81859-3"}]': finished 2024-02-14T17:45:24.488 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: osdmap e108: 8 total, 8 up, 8 in 2024-02-14T17:45:24.488 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:24.488 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:24.488 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3557046394' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi178-80984-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:24.488 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: from='client.47873 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi178-80984-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:24.488 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: Running main() from gmock_main.cc 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [----------] Global test environment set-up. 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (19996 ms) 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (8471 ms) 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (11630 ms) 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [----------] 3 tests from NeoradosECList (40097 ms total) 2024-02-14T17:45:24.548 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: 2024-02-14T17:45:24.549 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [----------] Global test environment tear-down 2024-02-14T17:45:24.549 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (40097 ms total) 2024-02-14T17:45:24.549 INFO:tasks.workunit.client.0.smithi178.stdout: ec_list: [ PASSED ] 3 tests. 2024-02-14T17:45:24.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:24.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:24.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: pgmap v116: 552 pgs: 160 creating+peering, 360 active+clean, 32 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:45:24.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: pgmap v118: 648 pgs: 160 creating+peering, 360 active+clean, 128 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:45:24.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:24.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4"}]: dispatch 2024-02-14T17:45:24.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: from='client.38119 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:24.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi178-81859-3"}]': finished 2024-02-14T17:45:24.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: osdmap e108: 8 total, 8 up, 8 in 2024-02-14T17:45:24.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:24.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:24.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3557046394' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi178-80984-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:24.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: from='client.47873 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi178-80984-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:24.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:24.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:24.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:24 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: pgmap v116: 552 pgs: 160 creating+peering, 360 active+clean, 32 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:45:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: pgmap v118: 648 pgs: 160 creating+peering, 360 active+clean, 128 unknown; 145 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4"}]: dispatch 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: from='client.38119 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81367-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi178-81859-3"}]': finished 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: osdmap e108: 8 total, 8 up, 8 in 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1274744101' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: from='client.30280 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi178-81859-3"}]: dispatch 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3557046394' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi178-80984-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: from='client.47873 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi178-80984-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:24 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:26.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:45:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:45:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4"}]': finished 2024-02-14T17:45:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi178-81859-3"}]': finished 2024-02-14T17:45:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.47873 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi178-80984-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: osdmap e109: 8 total, 8 up, 8 in 2024-02-14T17:45:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/679803162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi178-80647-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.41596 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi178-80647-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.41596 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi178-80647-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6"}]': finished 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: osdmap e110: 8 total, 8 up, 8 in 2024-02-14T17:45:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3534213164' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi178-80663-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4"}]': finished 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi178-81859-3"}]': finished 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.47873 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi178-80984-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: osdmap e109: 8 total, 8 up, 8 in 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/679803162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi178-80647-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.41596 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi178-80647-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:26.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:26.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:26.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:26.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.41596 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi178-80647-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:26.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6"}]': finished 2024-02-14T17:45:26.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:26.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: osdmap e110: 8 total, 8 up, 8 in 2024-02-14T17:45:26.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3534213164' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi178-80663-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.42460 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi178-80663-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:26.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:26.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi178-81146-6", "pool2": "test-rados-api-smithi178-81146-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.087 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi178-81146-6", "pool2": "test-rados-api-smithi178-81146-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.087 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.42460 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi178-80663-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.087 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:26.087 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:26.087 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi178-81146-6", "pool2": "test-rados-api-smithi178-81146-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.087 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:25 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi178-81146-6", "pool2": "test-rados-api-smithi178-81146-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4"}]': finished 2024-02-14T17:45:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.30280 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi178-81859-3"}]': finished 2024-02-14T17:45:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.47873 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi178-80984-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: osdmap e109: 8 total, 8 up, 8 in 2024-02-14T17:45:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/679803162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi178-80647-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.41596 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi178-80647-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6"}]: dispatch 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.41596 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi178-80647-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi178-81146-4", "tierpool": "test-rados-api-smithi178-81146-6"}]': finished 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: osdmap e110: 8 total, 8 up, 8 in 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3534213164' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi178-80663-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.42460 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi178-80663-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:26.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3497975402' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi178-81146-6", "pool2": "test-rados-api-smithi178-81146-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:45:26.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:25 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi178-81146-6", "pool2": "test-rados-api-smithi178-81146-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:45:27.570 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: Running main() from gmock_main.cc 2024-02-14T17:45:27.570 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-02-14T17:45:27.570 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [----------] Global test environment set-up. 2024-02-14T17:45:27.570 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-02-14T17:45:27.570 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-02-14T17:45:27.570 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (10788 ms) 2024-02-14T17:45:27.570 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-02-14T17:45:27.571 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (5627 ms) 2024-02-14T17:45:27.571 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-02-14T17:45:27.571 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (4777 ms) 2024-02-14T17:45:27.571 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-02-14T17:45:27.571 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (5041 ms) 2024-02-14T17:45:27.571 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-02-14T17:45:27.571 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3765 ms) 2024-02-14T17:45:27.571 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-02-14T17:45:27.571 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3498 ms) 2024-02-14T17:45:27.572 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-02-14T17:45:27.572 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (9568 ms) 2024-02-14T17:45:27.572 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (43064 ms total) 2024-02-14T17:45:27.572 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: 2024-02-14T17:45:27.572 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [----------] Global test environment tear-down 2024-02-14T17:45:27.572 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (43064 ms total) 2024-02-14T17:45:27.572 INFO:tasks.workunit.client.0.smithi178.stdout: write_operations: [ PASSED ] 7 tests. 2024-02-14T17:45:27.582 INFO:tasks.workunit.client.0.smithi178.stdout:pi_watch_notify_pp: flushed 2024-02-14T17:45:27.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3050 ms) 2024-02-14T17:45:27.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-02-14T17:45:27.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: trying... 2024-02-14T17:45:27.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: handle_notify cookie 94491657587904 notify_id 425201762310 notifier_gid 35372 2024-02-14T17:45:27.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: timed out 2024-02-14T17:45:27.582 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: flushing 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: flushed 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3087 ms) 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: List watches 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: notify2 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: handle_notify cookie 94491657587904 notify_id 438086664199 notifier_gid 35372 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: notify2 done 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: watch_check 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: unwatch2 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: flushing 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: done 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3062 ms) 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: List watches 2024-02-14T17:45:27.583 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: notify2 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: handle_notify cookie 94491657587904 notify_id 450971566083 notifier_gid 35372 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: notify2 done 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: watch_check 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: unwatch2 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: flushing 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: done 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3037 ms) 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (17629 ms total) 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (43358 ms total) 2024-02-14T17:45:27.584 INFO:tasks.workunit.client.0.smithi178.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-02-14T17:45:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: pgmap v122: 616 pgs: 4 creating+activating, 30 creating+peering, 482 active+clean, 100 unknown; 145 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-14T17:45:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:45:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:45:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:45:27.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: 14.1 deep-scrub starts 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: 14.1 deep-scrub ok 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: 14.9 deep-scrub starts 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: 14.9 deep-scrub ok 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.42460 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi178-80663-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-9"}]': finished 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi178-81146-6", "pool2": "test-rados-api-smithi178-81146-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: osdmap e111: 8 total, 8 up, 8 in 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-9", "mode": "writeback"}]: dispatch 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-9", "mode": "writeback"}]: dispatch 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/230039968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.43144 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:27.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:27.723 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: pgmap v122: 616 pgs: 4 creating+activating, 30 creating+peering, 482 active+clean, 100 unknown; 145 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-14T17:45:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:45:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:45:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:45:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:45:27.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:45:27.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:45:27.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:45:27.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: 14.1 deep-scrub starts 2024-02-14T17:45:27.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: 14.1 deep-scrub ok 2024-02-14T17:45:27.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: 14.9 deep-scrub starts 2024-02-14T17:45:27.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: 14.9 deep-scrub ok 2024-02-14T17:45:27.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.42460 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi178-80663-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:27.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-9"}]': finished 2024-02-14T17:45:27.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi178-81146-6", "pool2": "test-rados-api-smithi178-81146-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T17:45:27.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: osdmap e111: 8 total, 8 up, 8 in 2024-02-14T17:45:27.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-9", "mode": "writeback"}]: dispatch 2024-02-14T17:45:27.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-9", "mode": "writeback"}]: dispatch 2024-02-14T17:45:27.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/230039968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:27.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.43144 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:27.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:27.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:27.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:27.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: pgmap v122: 616 pgs: 4 creating+activating, 30 creating+peering, 482 active+clean, 100 unknown; 145 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-14T17:45:27.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:27.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:45:27.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.0"}]: dispatch 2024-02-14T17:45:27.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:45:27.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.1"}]: dispatch 2024-02-14T17:45:27.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:45:27.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.2"}]: dispatch 2024-02-14T17:45:27.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:45:27.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.3"}]: dispatch 2024-02-14T17:45:27.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:45:27.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.4"}]: dispatch 2024-02-14T17:45:27.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.5"}]: dispatch 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.6"}]: dispatch 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.7"}]: dispatch 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.8"}]: dispatch 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "14.9"}]: dispatch 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: 14.1 deep-scrub starts 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: 14.1 deep-scrub ok 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: 14.9 deep-scrub starts 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: 14.9 deep-scrub ok 2024-02-14T17:45:27.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.42460 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi178-80663-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:27.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-9"}]': finished 2024-02-14T17:45:27.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.35372 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi178-81146-6", "pool2": "test-rados-api-smithi178-81146-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T17:45:27.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: osdmap e111: 8 total, 8 up, 8 in 2024-02-14T17:45:27.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-9", "mode": "writeback"}]: dispatch 2024-02-14T17:45:27.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-9", "mode": "writeback"}]: dispatch 2024-02-14T17:45:27.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/230039968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:27.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.43144 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:27.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:27.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:27.841 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:28.576 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-02-14T17:45:28.576 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-02-14T17:45:28.576 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-02-14T17:45:28.576 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-02-14T17:45:28.576 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (10977 ms) 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (6909 ms) 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (4822 ms) 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3711 ms) 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (5147 ms) 2024-02-14T17:45:28.577 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-02-14T17:45:28.578 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (4339 ms) 2024-02-14T17:45:28.578 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-02-14T17:45:28.578 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (8366 ms) 2024-02-14T17:45:28.578 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (44271 ms total) 2024-02-14T17:45:28.578 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: 2024-02-14T17:45:28.578 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-02-14T17:45:28.578 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (44271 ms total) 2024-02-14T17:45:28.578 INFO:tasks.workunit.client.0.smithi178.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-02-14T17:45:28.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: 14.0 deep-scrub starts 2024-02-14T17:45:28.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: 14.0 deep-scrub ok 2024-02-14T17:45:28.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: 14.8 deep-scrub starts 2024-02-14T17:45:28.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: 14.8 deep-scrub ok 2024-02-14T17:45:28.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: 14.2 deep-scrub starts 2024-02-14T17:45:28.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: 14.2 deep-scrub ok 2024-02-14T17:45:28.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: 14.7 deep-scrub starts 2024-02-14T17:45:28.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: 14.7 deep-scrub ok 2024-02-14T17:45:28.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:28.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-9", "mode": "writeback"}]': finished 2024-02-14T17:45:28.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: from='client.43144 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:28.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: osdmap e112: 8 total, 8 up, 8 in 2024-02-14T17:45:28.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4268200014' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi178-80647-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:28.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: from='client.52319 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi178-80647-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:28.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:28.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:28.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: 14.0 deep-scrub starts 2024-02-14T17:45:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: 14.0 deep-scrub ok 2024-02-14T17:45:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: 14.8 deep-scrub starts 2024-02-14T17:45:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: 14.8 deep-scrub ok 2024-02-14T17:45:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: 14.2 deep-scrub starts 2024-02-14T17:45:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: 14.2 deep-scrub ok 2024-02-14T17:45:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: 14.7 deep-scrub starts 2024-02-14T17:45:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: 14.7 deep-scrub ok 2024-02-14T17:45:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-9", "mode": "writeback"}]': finished 2024-02-14T17:45:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: from='client.43144 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: osdmap e112: 8 total, 8 up, 8 in 2024-02-14T17:45:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4268200014' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi178-80647-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: from='client.52319 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi178-80647-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:28.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: 14.0 deep-scrub starts 2024-02-14T17:45:28.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: 14.0 deep-scrub ok 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: 14.8 deep-scrub starts 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: 14.8 deep-scrub ok 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: 14.2 deep-scrub starts 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: 14.2 deep-scrub ok 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: 14.7 deep-scrub starts 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: 14.7 deep-scrub ok 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-9", "mode": "writeback"}]': finished 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: from='client.43144 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: osdmap e112: 8 total, 8 up, 8 in 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4268200014' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi178-80647-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: from='client.52319 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi178-80647-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:28.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:28.839 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:28.839 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:29.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: 14.3 deep-scrub starts 2024-02-14T17:45:29.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: 14.3 deep-scrub ok 2024-02-14T17:45:29.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: pgmap v124: 644 pgs: 32 creating+peering, 516 active+clean, 96 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 op/s 2024-02-14T17:45:29.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: 14.4 deep-scrub starts 2024-02-14T17:45:29.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: 14.4 deep-scrub ok 2024-02-14T17:45:29.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: 14.6 deep-scrub starts 2024-02-14T17:45:29.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: 14.6 deep-scrub ok 2024-02-14T17:45:29.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:29.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: from='client.52319 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi178-80647-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:29.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:45:29.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: osdmap e113: 8 total, 8 up, 8 in 2024-02-14T17:45:29.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:29.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:29.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2709152336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi178-80663-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:29.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: from='client.45310 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi178-80663-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:29.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:29.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:45:29.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: pgmap v127: 580 pgs: 420 active+clean, 160 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:29.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:29.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:29 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: 14.3 deep-scrub starts 2024-02-14T17:45:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: 14.3 deep-scrub ok 2024-02-14T17:45:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: pgmap v124: 644 pgs: 32 creating+peering, 516 active+clean, 96 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 op/s 2024-02-14T17:45:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: 14.4 deep-scrub starts 2024-02-14T17:45:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: 14.4 deep-scrub ok 2024-02-14T17:45:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: 14.6 deep-scrub starts 2024-02-14T17:45:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: 14.6 deep-scrub ok 2024-02-14T17:45:29.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: from='client.52319 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi178-80647-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: osdmap e113: 8 total, 8 up, 8 in 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2709152336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi178-80663-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: from='client.45310 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi178-80663-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: pgmap v127: 580 pgs: 420 active+clean, 160 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:29.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: 14.3 deep-scrub starts 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: 14.3 deep-scrub ok 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: pgmap v124: 644 pgs: 32 creating+peering, 516 active+clean, 96 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 op/s 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: 14.4 deep-scrub starts 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: 14.4 deep-scrub ok 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: 14.6 deep-scrub starts 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: 14.6 deep-scrub ok 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: from='client.52319 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi178-80647-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: osdmap e113: 8 total, 8 up, 8 in 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9"}]: dispatch 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2709152336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi178-80663-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:29.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: from='client.45310 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi178-80663-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:29.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:29.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:45:29.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: pgmap v127: 580 pgs: 420 active+clean, 160 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:29.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:29.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:29 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:30 smithi184 ceph-mon[33536]: 14.5 deep-scrub starts 2024-02-14T17:45:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:30 smithi184 ceph-mon[33536]: 14.5 deep-scrub ok 2024-02-14T17:45:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:30 smithi184 ceph-mon[33536]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-14T17:45:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:30 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:30 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9"}]': finished 2024-02-14T17:45:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:30 smithi184 ceph-mon[33536]: from='client.45310 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi178-80663-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:30 smithi184 ceph-mon[33536]: osdmap e114: 8 total, 8 up, 8 in 2024-02-14T17:45:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:30 smithi184 ceph-mon[33536]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:31.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[30321]: 14.5 deep-scrub starts 2024-02-14T17:45:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[30321]: 14.5 deep-scrub ok 2024-02-14T17:45:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[30321]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-14T17:45:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9"}]': finished 2024-02-14T17:45:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[30321]: from='client.45310 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi178-80663-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[30321]: osdmap e114: 8 total, 8 up, 8 in 2024-02-14T17:45:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[30321]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[39811]: 14.5 deep-scrub starts 2024-02-14T17:45:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[39811]: 14.5 deep-scrub ok 2024-02-14T17:45:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[39811]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-14T17:45:31.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:31.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-9"}]': finished 2024-02-14T17:45:31.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[39811]: from='client.45310 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi178-80663-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:31.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[39811]: osdmap e114: 8 total, 8 up, 8 in 2024-02-14T17:45:31.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:31.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:30 smithi178 ceph-mon[39811]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:31.380 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-02-14T17:45:31.380 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [----------] Global test environment set-up. 2024-02-14T17:45:31.380 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-02-14T17:45:31.380 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-02-14T17:45:31.380 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-02-14T17:45:31.380 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-14T17:45:31.380 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: 2024-02-14T17:45:31.380 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-02-14T17:45:31.380 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-02-14T17:45:31.380 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-02-14T17:45:31.380 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: 2024-02-14T17:44:44.190+0000 7f976dc48780 -1 failed for service _ceph-mon._tcp 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: 2024-02-14T17:44:44.190+0000 7f976dc48780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (51 ms) 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2010 ms) 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2062 ms total) 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: started 0x55b8965f7600 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: started 0x55b89692db50 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: started 2 aios 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: waiting 0x55b8965f7600 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: waiting 0x55b89692db50 2024-02-14T17:45:31.381 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: done. 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (13694 ms) 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (13694 ms total) 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMisc.Exec (122 ms) 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (7 ms) 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (3 ms) 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5708 ms) 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-02-14T17:45:31.382 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-02-14T17:45:31.383 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-02-14T17:45:31.383 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-02-14T17:45:31.383 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-02-14T17:45:31.383 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (23055 ms) 2024-02-14T17:45:31.383 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [----------] 8 tests from LibRadosMisc (28896 ms total) 2024-02-14T17:45:31.383 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: 2024-02-14T17:45:31.383 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [----------] Global test environment tear-down 2024-02-14T17:45:31.383 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (47204 ms total) 2024-02-14T17:45:31.383 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc: [ PASSED ] 12 tests. 2024-02-14T17:45:31.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:45:30 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:45:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:31 smithi184 ceph-mon[33536]: pgmap v130: 548 pgs: 28 creating+activating, 33 creating+peering, 435 active+clean, 52 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 924 B/s rd, 22 MiB/s wr, 3 op/s 2024-02-14T17:45:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:31 smithi184 ceph-mon[33536]: osdmap e115: 8 total, 8 up, 8 in 2024-02-14T17:45:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:31 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:31 smithi184 ceph-mon[33536]: osdmap e116: 8 total, 8 up, 8 in 2024-02-14T17:45:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1594362265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi178-80647-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:31 smithi184 ceph-mon[33536]: from='client.54989 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi178-80647-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:31 smithi184 ceph-mon[33536]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:32.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[30321]: pgmap v130: 548 pgs: 28 creating+activating, 33 creating+peering, 435 active+clean, 52 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 924 B/s rd, 22 MiB/s wr, 3 op/s 2024-02-14T17:45:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[30321]: osdmap e115: 8 total, 8 up, 8 in 2024-02-14T17:45:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[30321]: osdmap e116: 8 total, 8 up, 8 in 2024-02-14T17:45:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1594362265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi178-80647-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[30321]: from='client.54989 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi178-80647-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[30321]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:32.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[39811]: pgmap v130: 548 pgs: 28 creating+activating, 33 creating+peering, 435 active+clean, 52 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 924 B/s rd, 22 MiB/s wr, 3 op/s 2024-02-14T17:45:32.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[39811]: osdmap e115: 8 total, 8 up, 8 in 2024-02-14T17:45:32.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:32.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:32.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[39811]: osdmap e116: 8 total, 8 up, 8 in 2024-02-14T17:45:32.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1594362265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi178-80647-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:32.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:32.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[39811]: from='client.54989 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi178-80647-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:32.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:31 smithi178 ceph-mon[39811]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:32 smithi184 ceph-mon[33536]: from='client.54989 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi178-80647-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:32 smithi184 ceph-mon[33536]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:32 smithi184 ceph-mon[33536]: osdmap e117: 8 total, 8 up, 8 in 2024-02-14T17:45:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2574996583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi178-80663-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:32 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:32 smithi184 ceph-mon[33536]: from='client.48754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi178-80663-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:32 smithi184 ceph-mon[33536]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[39811]: from='client.54989 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi178-80647-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[39811]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[39811]: osdmap e117: 8 total, 8 up, 8 in 2024-02-14T17:45:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2574996583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi178-80663-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[39811]: from='client.48754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi178-80663-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[39811]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[30321]: from='client.54989 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi178-80647-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[30321]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-81009-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[30321]: osdmap e117: 8 total, 8 up, 8 in 2024-02-14T17:45:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2574996583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi178-80663-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[30321]: from='client.48754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi178-80663-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:32 smithi178 ceph-mon[30321]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: pgmap v133: 516 pgs: 8 creating+peering, 292 active+clean, 216 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.48754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi178-80663-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: osdmap e118: 8 total, 8 up, 8 in 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi178-80957-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:45:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi178-80957-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: pgmap v133: 516 pgs: 8 creating+peering, 292 active+clean, 216 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.48754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi178-80663-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: osdmap e118: 8 total, 8 up, 8 in 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi178-80957-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi178-80957-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: pgmap v133: 516 pgs: 8 creating+peering, 292 active+clean, 216 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.48754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi178-80663-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: osdmap e118: 8 total, 8 up, 8 in 2024-02-14T17:45:34.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:45:34.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi178-80957-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:45:34.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi178-80957-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:34.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:45:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:45:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[39811]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache", "force_nonempty":""}]': finished 2024-02-14T17:45:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[39811]: osdmap e119: 8 total, 8 up, 8 in 2024-02-14T17:45:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[39811]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2699445523' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi178-80647-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[39811]: pgmap v136: 420 pgs: 292 active+clean, 128 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:36.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[39811]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[30321]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache", "force_nonempty":""}]': finished 2024-02-14T17:45:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[30321]: osdmap e119: 8 total, 8 up, 8 in 2024-02-14T17:45:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[30321]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2699445523' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi178-80647-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[30321]: pgmap v136: 420 pgs: 292 active+clean, 128 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:36.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:35 smithi178 ceph-mon[30321]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:35 smithi184 ceph-mon[33536]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache", "force_nonempty":""}]': finished 2024-02-14T17:45:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:35 smithi184 ceph-mon[33536]: osdmap e119: 8 total, 8 up, 8 in 2024-02-14T17:45:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:35 smithi184 ceph-mon[33536]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2699445523' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi178-80647-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:35 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:35 smithi184 ceph-mon[33536]: pgmap v136: 420 pgs: 292 active+clean, 128 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:36.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:35 smithi184 ceph-mon[33536]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi178-80957-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]': finished 2024-02-14T17:45:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2699445523' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi178-80647-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache"}]: dispatch 2024-02-14T17:45:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: osdmap e120: 8 total, 8 up, 8 in 2024-02-14T17:45:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:37.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache"}]: dispatch 2024-02-14T17:45:37.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-11"}]': finished 2024-02-14T17:45:37.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache"}]': finished 2024-02-14T17:45:37.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: osdmap e121: 8 total, 8 up, 8 in 2024-02-14T17:45:37.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/702023511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:37.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-11", "mode": "writeback"}]: dispatch 2024-02-14T17:45:37.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.56312 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:37.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-11", "mode": "writeback"}]: dispatch 2024-02-14T17:45:37.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi178-80957-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]': finished 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2699445523' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi178-80647-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache"}]: dispatch 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: osdmap e120: 8 total, 8 up, 8 in 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache"}]: dispatch 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-11"}]': finished 2024-02-14T17:45:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache"}]': finished 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: osdmap e121: 8 total, 8 up, 8 in 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/702023511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-11", "mode": "writeback"}]: dispatch 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.56312 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-11", "mode": "writeback"}]: dispatch 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi178-80957-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]': finished 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-81009-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2699445523' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi178-80647-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:37.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/57183367' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache"}]: dispatch 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: osdmap e120: 8 total, 8 up, 8 in 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.55829 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache"}]: dispatch 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-11"}]': finished 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.55829 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-81009-10", "tierpool":"test-rados-api-smithi178-81009-10-cache"}]': finished 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: osdmap e121: 8 total, 8 up, 8 in 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/702023511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-11", "mode": "writeback"}]: dispatch 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.56312 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-11", "mode": "writeback"}]: dispatch 2024-02-14T17:45:37.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:37.393 INFO:tasks.workunit.client.0.smithi178.stdout: misc: Running main() from gmock_main.cc 2024-02-14T17:45:37.393 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-02-14T17:45:37.393 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [----------] Global test environment set-up. 2024-02-14T17:45:37.393 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-02-14T17:45:37.393 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-02-14T17:45:37.393 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.Version (10474 ms) 2024-02-14T17:45:37.393 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-02-14T17:45:37.393 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (3716 ms) 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.LongName (5021 ms) 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (4064 ms) 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (5204 ms) 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (5066 ms) 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.Exec (5516 ms) 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-02-14T17:45:37.394 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3043 ms) 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3002 ms) 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.BigObject (2784 ms) 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (2264 ms) 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (2772 ms) 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [----------] 12 tests from NeoRadosMisc (52926 ms total) 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [----------] Global test environment tear-down 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [==========] 12 tests from 1 test suite ran. (52926 ms total) 2024-02-14T17:45:37.395 INFO:tasks.workunit.client.0.smithi178.stdout: misc: [ PASSED ] 12 tests. 2024-02-14T17:45:38.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:37 smithi184 ceph-mon[33536]: pgmap v139: 556 pgs: 16 creating+peering, 388 active+clean, 152 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-02-14T17:45:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:37 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:37 smithi184 ceph-mon[33536]: from='client.56312 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:37 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-11", "mode": "writeback"}]': finished 2024-02-14T17:45:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:37 smithi184 ceph-mon[33536]: osdmap e122: 8 total, 8 up, 8 in 2024-02-14T17:45:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4203457234' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi178-80647-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:38.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[30321]: pgmap v139: 556 pgs: 16 creating+peering, 388 active+clean, 152 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-02-14T17:45:38.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:38.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[30321]: from='client.56312 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:38.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-11", "mode": "writeback"}]': finished 2024-02-14T17:45:38.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[30321]: osdmap e122: 8 total, 8 up, 8 in 2024-02-14T17:45:38.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4203457234' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi178-80647-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[39811]: pgmap v139: 556 pgs: 16 creating+peering, 388 active+clean, 152 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-02-14T17:45:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[39811]: from='client.56312 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-11", "mode": "writeback"}]': finished 2024-02-14T17:45:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[39811]: osdmap e122: 8 total, 8 up, 8 in 2024-02-14T17:45:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4203457234' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi178-80647-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:38.369 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: Running main() from gmock_main.cc 2024-02-14T17:45:38.369 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-02-14T17:45:38.369 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [----------] Global test environment set-up. 2024-02-14T17:45:38.369 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-02-14T17:45:38.369 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-02-14T17:45:38.369 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ OK ] LibRadosPools.PoolList (10772 ms) 2024-02-14T17:45:38.369 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-02-14T17:45:38.369 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3719 ms) 2024-02-14T17:45:38.369 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (5492 ms) 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3604 ms) 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (5199 ms) 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5064 ms) 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (7179 ms) 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5179 ms) 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7994 ms) 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [----------] 9 tests from LibRadosPools (54202 ms total) 2024-02-14T17:45:38.370 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: 2024-02-14T17:45:38.371 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [----------] Global test environment tear-down 2024-02-14T17:45:38.371 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (54203 ms total) 2024-02-14T17:45:38.371 INFO:tasks.workunit.client.0.smithi178.stdout: api_pool: [ PASSED ] 9 tests. 2024-02-14T17:45:39.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4203457234' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi178-80647-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:38 smithi184 ceph-mon[33536]: osdmap e123: 8 total, 8 up, 8 in 2024-02-14T17:45:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:38 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:39.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4203457234' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi178-80647-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[39811]: osdmap e123: 8 total, 8 up, 8 in 2024-02-14T17:45:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4203457234' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi178-80647-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[30321]: osdmap e123: 8 total, 8 up, 8 in 2024-02-14T17:45:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:45:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:40.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:39 smithi184 ceph-mon[33536]: pgmap v142: 460 pgs: 3 creating+peering, 324 active+clean, 133 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-02-14T17:45:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:39 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:45:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:39 smithi184 ceph-mon[33536]: osdmap e124: 8 total, 8 up, 8 in 2024-02-14T17:45:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3088068748' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi178-80663-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:39 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:39 smithi184 ceph-mon[33536]: from='client.56327 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi178-80663-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:40.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:40.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[39811]: pgmap v142: 460 pgs: 3 creating+peering, 324 active+clean, 133 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-02-14T17:45:40.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:45:40.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[39811]: osdmap e124: 8 total, 8 up, 8 in 2024-02-14T17:45:40.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:40.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3088068748' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi178-80663-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:40.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:40.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[39811]: from='client.56327 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi178-80663-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:40.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:40.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[30321]: pgmap v142: 460 pgs: 3 creating+peering, 324 active+clean, 133 unknown; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-02-14T17:45:40.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:45:40.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[30321]: osdmap e124: 8 total, 8 up, 8 in 2024-02-14T17:45:40.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:40.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3088068748' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi178-80663-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:40.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11"}]: dispatch 2024-02-14T17:45:40.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[30321]: from='client.56327 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi178-80663-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:40.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:41.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:45:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:45:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:40 smithi184 ceph-mon[33536]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:40 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:40 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11"}]': finished 2024-02-14T17:45:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:40 smithi184 ceph-mon[33536]: from='client.56327 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi178-80663-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:40 smithi184 ceph-mon[33536]: osdmap e125: 8 total, 8 up, 8 in 2024-02-14T17:45:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/670097048' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi178-80647-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:41.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:41.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[39811]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:41.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:41.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11"}]': finished 2024-02-14T17:45:41.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[39811]: from='client.56327 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi178-80663-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:41.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[39811]: osdmap e125: 8 total, 8 up, 8 in 2024-02-14T17:45:41.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/670097048' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi178-80647-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:41.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:41.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[30321]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:41.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:45:41.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-11"}]': finished 2024-02-14T17:45:41.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[30321]: from='client.56327 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi178-80663-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:41.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[30321]: osdmap e125: 8 total, 8 up, 8 in 2024-02-14T17:45:41.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/670097048' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi178-80647-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:41.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:42.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:42 smithi184 ceph-mon[33536]: pgmap v145: 428 pgs: 9 creating+activating, 30 creating+peering, 378 active+clean, 11 unknown; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 260 KiB/s wr, 10 op/s 2024-02-14T17:45:42.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:42 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:42.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/670097048' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi178-80647-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:42.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:42 smithi184 ceph-mon[33536]: osdmap e126: 8 total, 8 up, 8 in 2024-02-14T17:45:42.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:42.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:42 smithi178 ceph-mon[39811]: pgmap v145: 428 pgs: 9 creating+activating, 30 creating+peering, 378 active+clean, 11 unknown; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 260 KiB/s wr, 10 op/s 2024-02-14T17:45:42.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:42 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:42.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/670097048' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi178-80647-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:42.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:42 smithi178 ceph-mon[39811]: osdmap e126: 8 total, 8 up, 8 in 2024-02-14T17:45:42.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:42.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:42 smithi178 ceph-mon[30321]: pgmap v145: 428 pgs: 9 creating+activating, 30 creating+peering, 378 active+clean, 11 unknown; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.7 KiB/s rd, 260 KiB/s wr, 10 op/s 2024-02-14T17:45:42.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:42 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:42.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/670097048' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi178-80647-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:42.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:42 smithi178 ceph-mon[30321]: osdmap e126: 8 total, 8 up, 8 in 2024-02-14T17:45:42.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:43.375 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: Running main() from gmock_main.cc 2024-02-14T17:45:43.375 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-02-14T17:45:43.375 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [----------] Global test environment set-up. 2024-02-14T17:45:43.375 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-02-14T17:45:43.375 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-02-14T17:45:43.375 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (10809 ms) 2024-02-14T17:45:43.375 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-02-14T17:45:43.375 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (5611 ms) 2024-02-14T17:45:43.375 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-02-14T17:45:43.376 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (3548 ms) 2024-02-14T17:45:43.376 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-02-14T17:45:43.376 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (3306 ms) 2024-02-14T17:45:43.376 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-02-14T17:45:43.376 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (4052 ms) 2024-02-14T17:45:43.376 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-02-14T17:45:43.376 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (4480 ms) 2024-02-14T17:45:43.376 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-02-14T17:45:43.376 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (4931 ms) 2024-02-14T17:45:43.376 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-02-14T17:45:43.376 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (3989 ms) 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (3373 ms) 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (2779 ms) 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (3268 ms) 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (2752 ms) 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (2989 ms) 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-02-14T17:45:43.377 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3011 ms) 2024-02-14T17:45:43.378 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (58899 ms total) 2024-02-14T17:45:43.378 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: 2024-02-14T17:45:43.378 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [----------] Global test environment tear-down 2024-02-14T17:45:43.378 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (58899 ms total) 2024-02-14T17:45:43.378 INFO:tasks.workunit.client.0.smithi178.stdout: read_operations: [ PASSED ] 14 tests. 2024-02-14T17:45:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:43 smithi184 ceph-mon[33536]: osdmap e127: 8 total, 8 up, 8 in 2024-02-14T17:45:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2305387192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi178-80663-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:43 smithi184 ceph-mon[33536]: from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi178-80663-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:43 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:43 smithi184 ceph-mon[33536]: pgmap v148: 460 pgs: 300 active+clean, 160 unknown; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-02-14T17:45:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[39811]: osdmap e127: 8 total, 8 up, 8 in 2024-02-14T17:45:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2305387192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi178-80663-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[39811]: from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi178-80663-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[39811]: pgmap v148: 460 pgs: 300 active+clean, 160 unknown; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-02-14T17:45:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[30321]: osdmap e127: 8 total, 8 up, 8 in 2024-02-14T17:45:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2305387192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi178-80663-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[30321]: from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi178-80663-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[30321]: pgmap v148: 460 pgs: 300 active+clean, 160 unknown; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-02-14T17:45:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:44.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:44 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:44.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:44 smithi184 ceph-mon[33536]: from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi178-80663-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:44.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:44 smithi184 ceph-mon[33536]: osdmap e128: 8 total, 8 up, 8 in 2024-02-14T17:45:44.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2046588277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi178-80647-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:44.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:44 smithi184 ceph-mon[33536]: from='client.56354 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi178-80647-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:44.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:44.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:44 smithi184 ceph-mon[33536]: from='client.56354 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi178-80647-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:44.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:44 smithi184 ceph-mon[33536]: osdmap e129: 8 total, 8 up, 8 in 2024-02-14T17:45:44.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[39811]: from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi178-80663-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[39811]: osdmap e128: 8 total, 8 up, 8 in 2024-02-14T17:45:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2046588277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi178-80647-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[39811]: from='client.56354 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi178-80647-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[39811]: from='client.56354 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi178-80647-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[39811]: osdmap e129: 8 total, 8 up, 8 in 2024-02-14T17:45:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[30321]: from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi178-80663-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[30321]: osdmap e128: 8 total, 8 up, 8 in 2024-02-14T17:45:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2046588277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi178-80647-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[30321]: from='client.56354 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi178-80647-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[30321]: from='client.56354 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi178-80647-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:44 smithi178 ceph-mon[30321]: osdmap e129: 8 total, 8 up, 8 in 2024-02-14T17:45:45.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[39811]: pgmap v151: 428 pgs: 300 active+clean, 128 unknown; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:45.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:45.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[39811]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:45.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[39811]: osdmap e130: 8 total, 8 up, 8 in 2024-02-14T17:45:45.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:45.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1430422522' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi178-80663-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:45.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[39811]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:45.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[30321]: pgmap v151: 428 pgs: 300 active+clean, 128 unknown; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:45.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:45.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[30321]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:45.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[30321]: osdmap e130: 8 total, 8 up, 8 in 2024-02-14T17:45:45.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:45.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1430422522' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi178-80663-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:45.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-mon[30321]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:45.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:45 smithi184 ceph-mon[33536]: pgmap v151: 428 pgs: 300 active+clean, 128 unknown; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:45.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:45.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:45 smithi184 ceph-mon[33536]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:45.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:45 smithi184 ceph-mon[33536]: osdmap e130: 8 total, 8 up, 8 in 2024-02-14T17:45:45.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:45.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1430422522' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi178-80663-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:45.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:45 smithi184 ceph-mon[33536]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:45.957 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:45:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:45:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:45:46.384 INFO:tasks.workunit.client.0.smithi178.stdout: io: Running main() from gmock_main.cc 2024-02-14T17:45:46.384 INFO:tasks.workunit.client.0.smithi178.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-02-14T17:45:46.384 INFO:tasks.workunit.client.0.smithi178.stdout: io: [----------] Global test environment set-up. 2024-02-14T17:45:46.384 INFO:tasks.workunit.client.0.smithi178.stdout: io: [----------] 14 tests from NeoRadosIo 2024-02-14T17:45:46.384 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-02-14T17:45:46.384 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.Limits (11877 ms) 2024-02-14T17:45:46.384 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-02-14T17:45:46.384 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (7828 ms) 2024-02-14T17:45:46.384 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-02-14T17:45:46.384 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.ReadOp (3591 ms) 2024-02-14T17:45:46.384 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.SparseRead (4062 ms) 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.RoundTrip (4130 ms) 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (5285 ms) 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3982 ms) 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3395 ms) 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (2762 ms) 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.Trunc (3263 ms) 2024-02-14T17:45:46.385 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.Remove (2748 ms) 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (3002 ms) 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.RmXattr (3010 ms) 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3004 ms) 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: [----------] 14 tests from NeoRadosIo (61940 ms total) 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: [----------] Global test environment tear-down 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: [==========] 14 tests from 1 test suite ran. (61940 ms total) 2024-02-14T17:45:46.386 INFO:tasks.workunit.client.0.smithi178.stdout: io: [ PASSED ] 14 tests. 2024-02-14T17:45:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:46.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1430422522' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi178-80663-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:46.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:46 smithi184 ceph-mon[33536]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]': finished 2024-02-14T17:45:46.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:46.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:46 smithi184 ceph-mon[33536]: osdmap e131: 8 total, 8 up, 8 in 2024-02-14T17:45:46.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:46 smithi184 ceph-mon[33536]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:46.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/297292102' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi178-80647-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:46.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1430422522' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi178-80663-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[39811]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]': finished 2024-02-14T17:45:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[39811]: osdmap e131: 8 total, 8 up, 8 in 2024-02-14T17:45:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[39811]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/297292102' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi178-80647-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1430422522' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi178-80663-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[30321]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi178-80957-10"}]': finished 2024-02-14T17:45:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2248692124' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:46.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[30321]: osdmap e131: 8 total, 8 up, 8 in 2024-02-14T17:45:46.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[30321]: from='client.56276 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]: dispatch 2024-02-14T17:45:46.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/297292102' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi178-80647-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:47.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:47 smithi184 ceph-mon[33536]: pgmap v154: 420 pgs: 22 creating+peering, 356 active+clean, 42 unknown; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 KiB/s wr, 55 op/s 2024-02-14T17:45:47.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:47.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:47 smithi184 ceph-mon[33536]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]': finished 2024-02-14T17:45:47.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/297292102' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi178-80647-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:47.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:47 smithi184 ceph-mon[33536]: osdmap e132: 8 total, 8 up, 8 in 2024-02-14T17:45:47.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:47 smithi178 ceph-mon[30321]: pgmap v154: 420 pgs: 22 creating+peering, 356 active+clean, 42 unknown; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 KiB/s wr, 55 op/s 2024-02-14T17:45:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:47 smithi178 ceph-mon[30321]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]': finished 2024-02-14T17:45:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/297292102' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi178-80647-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:47 smithi178 ceph-mon[30321]: osdmap e132: 8 total, 8 up, 8 in 2024-02-14T17:45:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:47 smithi178 ceph-mon[39811]: pgmap v154: 420 pgs: 22 creating+peering, 356 active+clean, 42 unknown; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 KiB/s wr, 55 op/s 2024-02-14T17:45:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:47 smithi178 ceph-mon[39811]: from='client.56276 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi178-80957-10"}]': finished 2024-02-14T17:45:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/297292102' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi178-80647-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:47 smithi178 ceph-mon[39811]: osdmap e132: 8 total, 8 up, 8 in 2024-02-14T17:45:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: osdmap e133: 8 total, 8 up, 8 in 2024-02-14T17:45:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:45:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:45:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2595748951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi178-80663-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:48 smithi184 ceph-mon[33536]: from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi178-80663-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: osdmap e133: 8 total, 8 up, 8 in 2024-02-14T17:45:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:45:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:45:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: osdmap e133: 8 total, 8 up, 8 in 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:45:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2595748951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi178-80663-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:49.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[30321]: from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi178-80663-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:49.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2595748951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi178-80663-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:49.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:48 smithi178 ceph-mon[39811]: from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi178-80663-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:49 smithi184 ceph-mon[33536]: pgmap v157: 388 pgs: 356 active+clean, 32 unknown; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 KiB/s wr, 55 op/s 2024-02-14T17:45:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:49 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-13"}]': finished 2024-02-14T17:45:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:49 smithi184 ceph-mon[33536]: from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi178-80663-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-13", "mode": "writeback"}]: dispatch 2024-02-14T17:45:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:49 smithi184 ceph-mon[33536]: osdmap e134: 8 total, 8 up, 8 in 2024-02-14T17:45:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1952531264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi178-80647-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:49 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-13", "mode": "writeback"}]: dispatch 2024-02-14T17:45:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:49 smithi184 ceph-mon[33536]: from='client.48871 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi178-80647-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:50.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[30321]: pgmap v157: 388 pgs: 356 active+clean, 32 unknown; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 KiB/s wr, 55 op/s 2024-02-14T17:45:50.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:50.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-13"}]': finished 2024-02-14T17:45:50.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[30321]: from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi178-80663-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:50.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-13", "mode": "writeback"}]: dispatch 2024-02-14T17:45:50.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[30321]: osdmap e134: 8 total, 8 up, 8 in 2024-02-14T17:45:50.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1952531264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi178-80647-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:50.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-13", "mode": "writeback"}]: dispatch 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[30321]: from='client.48871 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi178-80647-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[39811]: pgmap v157: 388 pgs: 356 active+clean, 32 unknown; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 KiB/s wr, 55 op/s 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-13"}]': finished 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[39811]: from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi178-80663-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-13", "mode": "writeback"}]: dispatch 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[39811]: osdmap e134: 8 total, 8 up, 8 in 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1952531264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi178-80647-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-13", "mode": "writeback"}]: dispatch 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[39811]: from='client.48871 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi178-80647-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:50.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:51.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:45:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:50 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:50 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:50 smithi184 ceph-mon[33536]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]': finished 2024-02-14T17:45:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:50 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-13", "mode": "writeback"}]': finished 2024-02-14T17:45:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:50 smithi184 ceph-mon[33536]: from='client.48871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi178-80647-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:50 smithi184 ceph-mon[33536]: osdmap e135: 8 total, 8 up, 8 in 2024-02-14T17:45:51.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:51.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:51.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:51.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[30321]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]': finished 2024-02-14T17:45:51.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-13", "mode": "writeback"}]': finished 2024-02-14T17:45:51.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[30321]: from='client.48871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi178-80647-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:51.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[30321]: osdmap e135: 8 total, 8 up, 8 in 2024-02-14T17:45:51.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:51.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:51.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:45:51.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[39811]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi178-80957-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]': finished 2024-02-14T17:45:51.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-13", "mode": "writeback"}]': finished 2024-02-14T17:45:51.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[39811]: from='client.48871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi178-80647-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:51.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[39811]: osdmap e135: 8 total, 8 up, 8 in 2024-02-14T17:45:51.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: pgmap v160: 396 pgs: 6 creating+activating, 30 creating+peering, 345 active+clean, 15 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-02-14T17:45:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:45:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:45:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:45:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:45:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:45:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:45:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:45:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: 170.8 scrub starts 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: 170.8 scrub ok 2024-02-14T17:45:52.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:52.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: osdmap e136: 8 total, 8 up, 8 in 2024-02-14T17:45:52.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1685851320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi178-80663-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:52.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.56402 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi178-80663-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:52.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: pgmap v160: 396 pgs: 6 creating+activating, 30 creating+peering, 345 active+clean, 15 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-02-14T17:45:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:45:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:45:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:45:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:45:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:45:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:45:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:45:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: 170.8 scrub starts 2024-02-14T17:45:52.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: 170.8 scrub ok 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: osdmap e136: 8 total, 8 up, 8 in 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1685851320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi178-80663-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.56402 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi178-80663-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: pgmap v160: 396 pgs: 6 creating+activating, 30 creating+peering, 345 active+clean, 15 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:45:52.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:45:52.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: 170.8 scrub starts 2024-02-14T17:45:52.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: 170.8 scrub ok 2024-02-14T17:45:52.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:52.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: osdmap e136: 8 total, 8 up, 8 in 2024-02-14T17:45:52.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1685851320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi178-80663-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:52.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.56402 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi178-80663-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:52.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:53.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: 170.6 scrub starts 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: 170.6 scrub ok 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: 170.2 deep-scrub starts 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: 170.2 deep-scrub ok 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: 170.7 scrub starts 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: 170.7 scrub ok 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: 170.9 scrub starts 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: 170.9 scrub ok 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: 170.3 scrub starts 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: 170.3 scrub ok 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: from='client.56402 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi178-80663-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: osdmap e137: 8 total, 8 up, 8 in 2024-02-14T17:45:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4208741339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi178-80647-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:53.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: 170.6 scrub starts 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: 170.6 scrub ok 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: 170.2 deep-scrub starts 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: 170.2 deep-scrub ok 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: 170.7 scrub starts 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: 170.7 scrub ok 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: 170.9 scrub starts 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: 170.9 scrub ok 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: 170.3 scrub starts 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: 170.3 scrub ok 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: from='client.56402 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi178-80663-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:53.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: osdmap e137: 8 total, 8 up, 8 in 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4208741339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi178-80647-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: 170.6 scrub starts 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: 170.6 scrub ok 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: 170.2 deep-scrub starts 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: 170.2 deep-scrub ok 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: 170.7 scrub starts 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: 170.7 scrub ok 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: 170.9 scrub starts 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: 170.9 scrub ok 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: 170.3 scrub starts 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: 170.3 scrub ok 2024-02-14T17:45:53.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: from='client.56402 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi178-80663-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: osdmap e137: 8 total, 8 up, 8 in 2024-02-14T17:45:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4208741339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi178-80647-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:53.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:54.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:53 smithi184 ceph-mon[33536]: 170.4 deep-scrub starts 2024-02-14T17:45:54.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:53 smithi184 ceph-mon[33536]: 170.4 deep-scrub ok 2024-02-14T17:45:54.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:53 smithi184 ceph-mon[33536]: pgmap v163: 428 pgs: 25 creating+peering, 324 active+clean, 79 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-02-14T17:45:54.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:53 smithi184 ceph-mon[33536]: 170.0 scrub starts 2024-02-14T17:45:54.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:53 smithi184 ceph-mon[33536]: 170.0 scrub ok 2024-02-14T17:45:54.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:53 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4208741339' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi178-80647-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:54.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:53 smithi184 ceph-mon[33536]: osdmap e138: 8 total, 8 up, 8 in 2024-02-14T17:45:54.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:53 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:54.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[39811]: 170.4 deep-scrub starts 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[39811]: 170.4 deep-scrub ok 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[39811]: pgmap v163: 428 pgs: 25 creating+peering, 324 active+clean, 79 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[39811]: 170.0 scrub starts 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[39811]: 170.0 scrub ok 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4208741339' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi178-80647-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[39811]: osdmap e138: 8 total, 8 up, 8 in 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[30321]: 170.4 deep-scrub starts 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[30321]: 170.4 deep-scrub ok 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[30321]: pgmap v163: 428 pgs: 25 creating+peering, 324 active+clean, 79 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[30321]: 170.0 scrub starts 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[30321]: 170.0 scrub ok 2024-02-14T17:45:54.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4208741339' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi178-80647-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:54.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[30321]: osdmap e138: 8 total, 8 up, 8 in 2024-02-14T17:45:54.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:53 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:55.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:54 smithi184 ceph-mon[33536]: 170.1 scrub starts 2024-02-14T17:45:55.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:54 smithi184 ceph-mon[33536]: 170.1 scrub ok 2024-02-14T17:45:55.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:54 smithi184 ceph-mon[33536]: 170.5 scrub starts 2024-02-14T17:45:55.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:54 smithi184 ceph-mon[33536]: 170.5 scrub ok 2024-02-14T17:45:55.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:54 smithi184 ceph-mon[33536]: osdmap e139: 8 total, 8 up, 8 in 2024-02-14T17:45:55.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4056137089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi178-80663-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:55.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:54 smithi184 ceph-mon[33536]: from='client.48892 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi178-80663-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:55.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:55.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[39811]: 170.1 scrub starts 2024-02-14T17:45:55.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[39811]: 170.1 scrub ok 2024-02-14T17:45:55.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[39811]: 170.5 scrub starts 2024-02-14T17:45:55.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[39811]: 170.5 scrub ok 2024-02-14T17:45:55.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[39811]: osdmap e139: 8 total, 8 up, 8 in 2024-02-14T17:45:55.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4056137089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi178-80663-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:55.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[39811]: from='client.48892 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi178-80663-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:55.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:55.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[30321]: 170.1 scrub starts 2024-02-14T17:45:55.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[30321]: 170.1 scrub ok 2024-02-14T17:45:55.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[30321]: 170.5 scrub starts 2024-02-14T17:45:55.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[30321]: 170.5 scrub ok 2024-02-14T17:45:55.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[30321]: osdmap e139: 8 total, 8 up, 8 in 2024-02-14T17:45:55.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4056137089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi178-80663-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:55.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[30321]: from='client.48892 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi178-80663-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:55.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:56.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:45:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:45:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[39811]: pgmap v166: 396 pgs: 25 creating+peering, 324 active+clean, 47 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[39811]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[39811]: from='client.48892 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi178-80663-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[39811]: osdmap e140: 8 total, 8 up, 8 in 2024-02-14T17:45:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/913383282' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi178-80647-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[39811]: from='client.48898 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi178-80647-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:56.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[30321]: pgmap v166: 396 pgs: 25 creating+peering, 324 active+clean, 47 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:56.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[30321]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:56.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[30321]: from='client.48892 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi178-80663-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:56.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[30321]: osdmap e140: 8 total, 8 up, 8 in 2024-02-14T17:45:56.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/913383282' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi178-80647-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:56.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[30321]: from='client.48898 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi178-80647-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:56.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:56.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:55 smithi184 ceph-mon[33536]: pgmap v166: 396 pgs: 25 creating+peering, 324 active+clean, 47 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:45:56.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:55 smithi184 ceph-mon[33536]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:56.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:55 smithi184 ceph-mon[33536]: from='client.48892 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi178-80663-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:56.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:55 smithi184 ceph-mon[33536]: osdmap e140: 8 total, 8 up, 8 in 2024-02-14T17:45:56.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/913383282' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi178-80647-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:56.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:55 smithi184 ceph-mon[33536]: from='client.48898 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi178-80647-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:56.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:57.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-14T17:45:57.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:56 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:56 smithi184 ceph-mon[33536]: from='client.48898 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi178-80647-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:57.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:56 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:57.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:56 smithi184 ceph-mon[33536]: osdmap e141: 8 total, 8 up, 8 in 2024-02-14T17:45:57.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2"}]: dispatch 2024-02-14T17:45:57.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:56 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:57.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[39811]: from='client.48898 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi178-80647-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[39811]: osdmap e141: 8 total, 8 up, 8 in 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2"}]: dispatch 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[30321]: from='client.48898 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi178-80647-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:57.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:57.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[30321]: osdmap e141: 8 total, 8 up, 8 in 2024-02-14T17:45:57.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2"}]: dispatch 2024-02-14T17:45:57.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:57.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:58.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:57 smithi184 ceph-mon[33536]: pgmap v169: 396 pgs: 22 creating+peering, 332 active+clean, 42 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 58 KiB/s rd, 4.0 KiB/s wr, 76 op/s 2024-02-14T17:45:58.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:57 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:58.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:57 smithi184 ceph-mon[33536]: osdmap e142: 8 total, 8 up, 8 in 2024-02-14T17:45:58.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3794157458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi178-80663-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:58.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:58.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:58.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:57 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:58.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:57 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:58.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:57 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:58.222 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[30321]: pgmap v169: 396 pgs: 22 creating+peering, 332 active+clean, 42 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 58 KiB/s rd, 4.0 KiB/s wr, 76 op/s 2024-02-14T17:45:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[30321]: osdmap e142: 8 total, 8 up, 8 in 2024-02-14T17:45:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3794157458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi178-80663-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:58.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[39811]: pgmap v169: 396 pgs: 22 creating+peering, 332 active+clean, 42 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 58 KiB/s rd, 4.0 KiB/s wr, 76 op/s 2024-02-14T17:45:58.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi178-80767-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:58.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[39811]: osdmap e142: 8 total, 8 up, 8 in 2024-02-14T17:45:58.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3794157458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi178-80663-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:58.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:58.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:58.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:45:58.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:45:58.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:58.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3794157458' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi178-80663-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:59 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T17:45:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:59 smithi184 ceph-mon[33536]: osdmap e143: 8 total, 8 up, 8 in 2024-02-14T17:45:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2239414919' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi178-80647-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:59 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:59 smithi184 ceph-mon[33536]: from='client.56435 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi178-80647-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:59 smithi184 ceph-mon[33536]: pgmap v172: 428 pgs: 332 active+clean, 96 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 58 KiB/s rd, 4.0 KiB/s wr, 76 op/s 2024-02-14T17:45:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:45:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:59.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3794157458' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi178-80663-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[39811]: osdmap e143: 8 total, 8 up, 8 in 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2239414919' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi178-80647-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[39811]: from='client.56435 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi178-80647-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[39811]: pgmap v172: 428 pgs: 332 active+clean, 96 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 58 KiB/s rd, 4.0 KiB/s wr, 76 op/s 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3794157458' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi178-80663-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[30321]: osdmap e143: 8 total, 8 up, 8 in 2024-02-14T17:45:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2239414919' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi178-80647-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:45:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[30321]: from='client.56435 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi178-80647-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:45:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[30321]: pgmap v172: 428 pgs: 332 active+clean, 96 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 58 KiB/s rd, 4.0 KiB/s wr, 76 op/s 2024-02-14T17:45:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:45:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:00 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T17:46:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:00 smithi184 ceph-mon[33536]: from='client.56435 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi178-80647-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:00 smithi184 ceph-mon[33536]: osdmap e144: 8 total, 8 up, 8 in 2024-02-14T17:46:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:46:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:00 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:46:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:00 smithi184 ceph-mon[33536]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T17:46:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[39811]: from='client.56435 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi178-80647-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[39811]: osdmap e144: 8 total, 8 up, 8 in 2024-02-14T17:46:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:46:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:46:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[39811]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T17:46:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[30321]: from='client.56435 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi178-80647-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[30321]: osdmap e144: 8 total, 8 up, 8 in 2024-02-14T17:46:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/709440843' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:46:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:46:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[30321]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:01.388 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:46:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:46:01.402 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: Running main() from gmock_main.cc 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [----------] Global test environment set-up. 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (5767 ms) 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (3532 ms) 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (3486 ms) 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (3668 ms) 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (16453 ms total) 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-02-14T17:46:01.403 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (5568 ms) 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (7033 ms) 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4733 ms) 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (17335 ms total) 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3008 ms) 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1990 ms) 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2009 ms) 2024-02-14T17:46:01.404 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2009 ms) 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9017 ms total) 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4040 ms) 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4002 ms) 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8042 ms total) 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [----------] Global test environment tear-down 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (77250 ms total) 2024-02-14T17:46:01.405 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-02-14T17:46:01.411 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-02-14T17:46:01.411 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-02-14T17:46:01.411 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-02-14T17:46:01.411 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-02-14T17:46:01.411 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-02-14T17:46:01.411 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-14T17:46:01.411 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: 2024-02-14T17:46:01.411 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: seed 80767 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (8 ms) 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (318 ms) 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (21 ms) 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (7 ms) 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (8 ms) 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (3 ms) 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-02-14T17:46:01.412 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (9 ms) 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (4 ms) 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (7 ms) 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: osd_max_attr_size = 0 2024-02-14T17:46:01.413 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1531 ms) 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (345 ms) 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: done waiting, doing copies 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: waiting for final deep scrubs... 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: done waiting 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60780 ms) 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (4 ms) 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-02-14T17:46:01.414 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (3 ms) 2024-02-14T17:46:01.415 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-02-14T17:46:01.415 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4438 ms) 2024-02-14T17:46:01.415 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-02-14T17:46:01.415 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-02-14T17:46:01.415 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-02-14T17:46:01.415 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-02-14T17:46:01.415 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-02-14T17:46:01.415 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-02-14T17:46:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:01 smithi184 ceph-mon[33536]: from='client.34703 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1"}]': finished 2024-02-14T17:46:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:01 smithi184 ceph-mon[33536]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]': finished 2024-02-14T17:46:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:01 smithi184 ceph-mon[33536]: osdmap e145: 8 total, 8 up, 8 in 2024-02-14T17:46:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/646368207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:01 smithi184 ceph-mon[33536]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:01 smithi184 ceph-mon[33536]: from='client.48922 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:01 smithi184 ceph-mon[33536]: pgmap v175: 388 pgs: 41 creating+peering, 324 active+clean, 23 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:46:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:01 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[39811]: from='client.34703 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1"}]': finished 2024-02-14T17:46:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[39811]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]': finished 2024-02-14T17:46:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[39811]: osdmap e145: 8 total, 8 up, 8 in 2024-02-14T17:46:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/646368207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[39811]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[39811]: from='client.48922 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[39811]: pgmap v175: 388 pgs: 41 creating+peering, 324 active+clean, 23 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:46:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:01.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[30321]: from='client.34703 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi178-80767-1","app":"app1","key":"key1"}]': finished 2024-02-14T17:46:01.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[30321]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]': finished 2024-02-14T17:46:01.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[30321]: osdmap e145: 8 total, 8 up, 8 in 2024-02-14T17:46:01.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/783561382' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:01.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/646368207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:01.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[30321]: from='client.48862 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]: dispatch 2024-02-14T17:46:01.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[30321]: from='client.48922 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:01.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[30321]: pgmap v175: 388 pgs: 41 creating+peering, 324 active+clean, 23 unknown; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:46:01.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:01.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:01 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]': finished 2024-02-14T17:46:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: from='client.48922 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: osdmap e146: 8 total, 8 up, 8 in 2024-02-14T17:46:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/930973636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80647-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80767-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:02.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80767-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:02.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]': finished 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: from='client.48922 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: osdmap e146: 8 total, 8 up, 8 in 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/930973636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80647-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80767-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80767-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: from='client.48862 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi178-80957-15"}]': finished 2024-02-14T17:46:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: from='client.48922 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: osdmap e146: 8 total, 8 up, 8 in 2024-02-14T17:46:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/930973636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80647-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80767-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80767-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/930973636' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80647-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:03.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:03 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80767-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:03.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:03 smithi184 ceph-mon[33536]: osdmap e147: 8 total, 8 up, 8 in 2024-02-14T17:46:03.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80767-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:03.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:03 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80767-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:03.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:03 smithi184 ceph-mon[33536]: pgmap v178: 356 pgs: 23 creating+peering, 292 active+clean, 41 unknown; 594 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:46:03.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:03.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/930973636' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80647-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80767-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[39811]: osdmap e147: 8 total, 8 up, 8 in 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80767-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80767-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[39811]: pgmap v178: 356 pgs: 23 creating+peering, 292 active+clean, 41 unknown; 594 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/930973636' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80647-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80767-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[30321]: osdmap e147: 8 total, 8 up, 8 in 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80767-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80767-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[30321]: pgmap v178: 356 pgs: 23 creating+peering, 292 active+clean, 41 unknown; 594 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:46:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:04.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:04 smithi184 ceph-mon[33536]: osdmap e148: 8 total, 8 up, 8 in 2024-02-14T17:46:04.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3389310622' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi178-80663-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:04.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:04.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:04 smithi178 ceph-mon[30321]: osdmap e148: 8 total, 8 up, 8 in 2024-02-14T17:46:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3389310622' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi178-80663-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:04 smithi178 ceph-mon[39811]: osdmap e148: 8 total, 8 up, 8 in 2024-02-14T17:46:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3389310622' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi178-80663-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:05.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80767-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80767-24"}]': finished 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3389310622' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi178-80663-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[39811]: osdmap e149: 8 total, 8 up, 8 in 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/292368553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi178-80647-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3137232797' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80984-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[39811]: from='client.56471 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi178-80647-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[39811]: from='client.56477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80984-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[39811]: pgmap v181: 396 pgs: 23 creating+peering, 260 active+clean, 113 unknown; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80767-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80767-24"}]': finished 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3389310622' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi178-80663-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[30321]: osdmap e149: 8 total, 8 up, 8 in 2024-02-14T17:46:05.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/292368553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi178-80647-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3137232797' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80984-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[30321]: from='client.56471 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi178-80647-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[30321]: from='client.56477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80984-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[30321]: pgmap v181: 396 pgs: 23 creating+peering, 260 active+clean, 113 unknown; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:46:05.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:05 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80767-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80767-24"}]': finished 2024-02-14T17:46:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3389310622' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi178-80663-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:05 smithi184 ceph-mon[33536]: osdmap e149: 8 total, 8 up, 8 in 2024-02-14T17:46:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/292368553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi178-80647-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3137232797' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80984-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:05 smithi184 ceph-mon[33536]: from='client.56471 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi178-80647-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:05 smithi184 ceph-mon[33536]: from='client.56477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80984-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:05.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:05 smithi184 ceph-mon[33536]: pgmap v181: 396 pgs: 23 creating+peering, 260 active+clean, 113 unknown; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:46:05.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:06.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:46:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:46:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:06 smithi184 ceph-mon[33536]: from='client.56471 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi178-80647-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:06 smithi184 ceph-mon[33536]: from='client.56477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80984-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:06 smithi184 ceph-mon[33536]: osdmap e150: 8 total, 8 up, 8 in 2024-02-14T17:46:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:06 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[30321]: from='client.56471 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi178-80647-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[30321]: from='client.56477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80984-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[30321]: osdmap e150: 8 total, 8 up, 8 in 2024-02-14T17:46:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[39811]: from='client.56471 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi178-80647-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[39811]: from='client.56477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80984-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[39811]: osdmap e150: 8 total, 8 up, 8 in 2024-02-14T17:46:06.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:06.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:06.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:07 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:07 smithi184 ceph-mon[33536]: osdmap e151: 8 total, 8 up, 8 in 2024-02-14T17:46:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3227510766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi178-80663-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:07 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:07 smithi184 ceph-mon[33536]: pgmap v184: 396 pgs: 6 creating+activating, 35 creating+peering, 292 active+clean, 63 unknown; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3227510766' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi178-80663-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:07 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[39811]: osdmap e151: 8 total, 8 up, 8 in 2024-02-14T17:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3227510766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi178-80663-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[39811]: pgmap v184: 396 pgs: 6 creating+activating, 35 creating+peering, 292 active+clean, 63 unknown; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3227510766' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi178-80663-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[30321]: osdmap e151: 8 total, 8 up, 8 in 2024-02-14T17:46:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3227510766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi178-80663-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[30321]: pgmap v184: 396 pgs: 6 creating+activating, 35 creating+peering, 292 active+clean, 63 unknown; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3227510766' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi178-80663-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:07 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80767-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:08 smithi184 ceph-mon[33536]: osdmap e152: 8 total, 8 up, 8 in 2024-02-14T17:46:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3022309540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi178-80647-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:08 smithi184 ceph-mon[33536]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:08 smithi184 ceph-mon[33536]: from='client.56498 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi178-80647-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:08 smithi184 ceph-mon[33536]: from='client.56498 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi178-80647-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:08 smithi184 ceph-mon[33536]: osdmap e153: 8 total, 8 up, 8 in 2024-02-14T17:46:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[39811]: osdmap e152: 8 total, 8 up, 8 in 2024-02-14T17:46:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3022309540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi178-80647-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[39811]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[39811]: from='client.56498 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi178-80647-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[39811]: from='client.56498 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi178-80647-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[39811]: osdmap e153: 8 total, 8 up, 8 in 2024-02-14T17:46:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[30321]: osdmap e152: 8 total, 8 up, 8 in 2024-02-14T17:46:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3022309540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi178-80647-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[30321]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[30321]: from='client.56498 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi178-80647-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[30321]: from='client.56498 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi178-80647-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:08 smithi178 ceph-mon[30321]: osdmap e153: 8 total, 8 up, 8 in 2024-02-14T17:46:09.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: pgmap v187: 332 pgs: 8 creating+peering, 292 active+clean, 32 unknown; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:46:09.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:09.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:09.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:09.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:09.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: osdmap e154: 8 total, 8 up, 8 in 2024-02-14T17:46:09.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:09.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:09.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3822436195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi178-80663-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:09.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.56516 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi178-80663-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:09.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi178-80984-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:09 smithi184 ceph-mon[33536]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi178-80984-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: pgmap v187: 332 pgs: 8 creating+peering, 292 active+clean, 32 unknown; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:46:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: osdmap e154: 8 total, 8 up, 8 in 2024-02-14T17:46:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:09.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:09.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3822436195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi178-80663-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:09.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.56516 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi178-80663-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:09.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi178-80984-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[39811]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi178-80984-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: pgmap v187: 332 pgs: 8 creating+peering, 292 active+clean, 32 unknown; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:46:09.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:09.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:09.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:09.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:09.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: osdmap e154: 8 total, 8 up, 8 in 2024-02-14T17:46:09.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:09.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:09.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3822436195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi178-80663-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:09.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.56516 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi178-80663-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:09.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi178-80984-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:09.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:09 smithi178 ceph-mon[30321]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi178-80984-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:10.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:10 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]': finished 2024-02-14T17:46:11.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:10 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]': finished 2024-02-14T17:46:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:10 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80767-24"}]': finished 2024-02-14T17:46:11.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:46:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: from='client.56516 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi178-80663-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: osdmap e155: 8 total, 8 up, 8 in 2024-02-14T17:46:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3938889359' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi178-80647-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: from='client.56525 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi178-80647-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: pgmap v190: 356 pgs: 15 creating+peering, 316 active+clean, 25 unknown; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:12.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi178-80984-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]': finished 2024-02-14T17:46:12.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]': finished 2024-02-14T17:46:12.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: from='client.56525 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi178-80647-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:12.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:11 smithi184 ceph-mon[33536]: osdmap e156: 8 total, 8 up, 8 in 2024-02-14T17:46:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: from='client.56516 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi178-80663-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: osdmap e155: 8 total, 8 up, 8 in 2024-02-14T17:46:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3938889359' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi178-80647-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: from='client.56525 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi178-80647-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: pgmap v190: 356 pgs: 15 creating+peering, 316 active+clean, 25 unknown; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi178-80984-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]': finished 2024-02-14T17:46:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]': finished 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: from='client.56525 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi178-80647-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[39811]: osdmap e156: 8 total, 8 up, 8 in 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: from='client.56516 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi178-80663-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1633294073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: osdmap e155: 8 total, 8 up, 8 in 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3938889359' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi178-80647-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]: dispatch 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: from='client.56525 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi178-80647-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: pgmap v190: 356 pgs: 15 creating+peering, 316 active+clean, 25 unknown; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:12.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi178-80984-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]': finished 2024-02-14T17:46:12.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: from='client.48931 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80767-24"}]': finished 2024-02-14T17:46:12.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: from='client.56525 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi178-80647-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:12.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:11 smithi178 ceph-mon[30321]: osdmap e156: 8 total, 8 up, 8 in 2024-02-14T17:46:13.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:12 smithi184 ceph-mon[33536]: osdmap e157: 8 total, 8 up, 8 in 2024-02-14T17:46:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3329524243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi178-80663-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/414773565' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi178-80767-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:12 smithi184 ceph-mon[33536]: from='client.48997 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi178-80663-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:12 smithi184 ceph-mon[33536]: from='client.49000 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi178-80767-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:12 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:13.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[30321]: osdmap e157: 8 total, 8 up, 8 in 2024-02-14T17:46:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3329524243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi178-80663-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/414773565' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi178-80767-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[30321]: from='client.48997 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi178-80663-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[30321]: from='client.49000 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi178-80767-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:13.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:13.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[39811]: osdmap e157: 8 total, 8 up, 8 in 2024-02-14T17:46:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3329524243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi178-80663-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/414773565' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi178-80767-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[39811]: from='client.48997 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi178-80663-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[39811]: from='client.49000 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi178-80767-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:13 smithi184 ceph-mon[33536]: pgmap v193: 364 pgs: 292 active+clean, 72 unknown; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:13 smithi184 ceph-mon[33536]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:13 smithi184 ceph-mon[33536]: from='client.48997 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi178-80663-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:13 smithi184 ceph-mon[33536]: from='client.49000 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi178-80767-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:13 smithi184 ceph-mon[33536]: osdmap e158: 8 total, 8 up, 8 in 2024-02-14T17:46:14.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3125669161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi178-80647-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:14.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:14.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[39811]: pgmap v193: 364 pgs: 292 active+clean, 72 unknown; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[39811]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[39811]: from='client.48997 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi178-80663-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[39811]: from='client.49000 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi178-80767-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[39811]: osdmap e158: 8 total, 8 up, 8 in 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3125669161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi178-80647-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[30321]: pgmap v193: 364 pgs: 292 active+clean, 72 unknown; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[30321]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[30321]: from='client.48997 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi178-80663-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[30321]: from='client.49000 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi178-80767-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[30321]: osdmap e158: 8 total, 8 up, 8 in 2024-02-14T17:46:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3125669161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi178-80647-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:14.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:15.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3125669161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi178-80647-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:15.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:15 smithi178 ceph-mon[39811]: osdmap e159: 8 total, 8 up, 8 in 2024-02-14T17:46:15.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:15 smithi178 ceph-mon[39811]: pgmap v196: 332 pgs: 292 active+clean, 40 unknown; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:15.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:15.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3125669161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi178-80647-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:15.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:15 smithi178 ceph-mon[30321]: osdmap e159: 8 total, 8 up, 8 in 2024-02-14T17:46:15.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:15 smithi178 ceph-mon[30321]: pgmap v196: 332 pgs: 292 active+clean, 40 unknown; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:15.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:15.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3125669161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi178-80647-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:15.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:15 smithi184 ceph-mon[33536]: osdmap e159: 8 total, 8 up, 8 in 2024-02-14T17:46:15.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:15 smithi184 ceph-mon[33536]: pgmap v196: 332 pgs: 292 active+clean, 40 unknown; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:15.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:16.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:46:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:46:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:16.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP (6 api_snapshots_pp: Running main() from gmock_main.cc 2024-02-14T17:46:16.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-02-14T17:46:16.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-02-14T17:46:16.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-02-14T17:46:16.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: seed 80984 2024-02-14T17:46:16.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-02-14T17:46:16.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (5767 ms) 2024-02-14T17:46:16.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-02-14T17:46:16.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (5492 ms) 2024-02-14T17:46:16.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-02-14T17:46:16.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (3602 ms) 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (4055 ms) 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (7821 ms) 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (26738 ms total) 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4147 ms) 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (4661 ms) 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6015 ms) 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4002 ms) 2024-02-14T17:46:16.389 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2025 ms) 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1430-g8727ce93/rpm/el9/BUILD/ceph-19.0.0-1430-g8727ce93/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi178-80984-7 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: waiting for snaps to purge 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17972 ms) 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (38822 ms total) 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (5 ms) 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (4818 ms) 2024-02-14T17:46:16.390 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (4823 ms total) 2024-02-14T17:46:16.391 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: 2024-02-14T17:46:16.391 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-02-14T17:46:16.391 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-02-14T17:46:16.391 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3010 ms) 2024-02-14T17:46:16.391 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-02-14T17:46:16.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:16 smithi184 ceph-mon[33536]: osdmap e160: 8 total, 8 up, 8 in 2024-02-14T17:46:16.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3534594308' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi178-80767-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:16.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3177031841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80663-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:16.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:16 smithi184 ceph-mon[33536]: from='client.49012 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi178-80767-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:16.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:16 smithi178 ceph-mon[30321]: osdmap e160: 8 total, 8 up, 8 in 2024-02-14T17:46:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3534594308' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi178-80767-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3177031841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80663-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:16 smithi178 ceph-mon[30321]: from='client.49012 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi178-80767-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:16.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:16 smithi178 ceph-mon[39811]: osdmap e160: 8 total, 8 up, 8 in 2024-02-14T17:46:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3534594308' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi178-80767-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3177031841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80663-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:16 smithi178 ceph-mon[39811]: from='client.49012 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi178-80767-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:17.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3177031841' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80663-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:17.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:17 smithi184 ceph-mon[33536]: from='client.49012 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi178-80767-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:17.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:17 smithi184 ceph-mon[33536]: osdmap e161: 8 total, 8 up, 8 in 2024-02-14T17:46:17.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3427071647' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi178-80647-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:17.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:17 smithi184 ceph-mon[33536]: from='client.56555 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi178-80647-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:17.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:17 smithi184 ceph-mon[33536]: pgmap v199: 396 pgs: 53 creating+peering, 300 active+clean, 43 unknown; 592 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:46:17.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3177031841' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80663-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[39811]: from='client.49012 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi178-80767-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[39811]: osdmap e161: 8 total, 8 up, 8 in 2024-02-14T17:46:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3427071647' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi178-80647-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[39811]: from='client.56555 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi178-80647-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:17.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[39811]: pgmap v199: 396 pgs: 53 creating+peering, 300 active+clean, 43 unknown; 592 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:46:17.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3177031841' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi178-80663-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[30321]: from='client.49012 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi178-80767-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[30321]: osdmap e161: 8 total, 8 up, 8 in 2024-02-14T17:46:17.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3427071647' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi178-80647-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:17.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[30321]: from='client.56555 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi178-80647-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:17.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[30321]: pgmap v199: 396 pgs: 53 creating+peering, 300 active+clean, 43 unknown; 592 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:46:17.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:18.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:18 smithi184 ceph-mon[33536]: from='client.56555 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi178-80647-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:18.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:18 smithi184 ceph-mon[33536]: osdmap e162: 8 total, 8 up, 8 in 2024-02-14T17:46:18.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:18.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:18 smithi178 ceph-mon[39811]: from='client.56555 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi178-80647-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:18 smithi178 ceph-mon[39811]: osdmap e162: 8 total, 8 up, 8 in 2024-02-14T17:46:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:18 smithi178 ceph-mon[30321]: from='client.56555 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi178-80647-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:18 smithi178 ceph-mon[30321]: osdmap e162: 8 total, 8 up, 8 in 2024-02-14T17:46:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:19.419 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosS snapshots: Running main() from gmock_main.cc 2024-02-14T17:46:19.419 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-02-14T17:46:19.419 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [----------] Global test environment set-up. 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (12941 ms) 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (8255 ms) 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (7260 ms) 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (8272 ms) 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (8360 ms) 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (45088 ms total) 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-02-14T17:46:19.420 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (5047 ms) 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (6747 ms) 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (8251 ms) 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (5752 ms) 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4015 ms) 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi178-81980-11. 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: Waiting for snaps to purge. 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20033 ms) 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (49846 ms total) 2024-02-14T17:46:19.421 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: 2024-02-14T17:46:19.422 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [----------] Global test environment tear-down 2024-02-14T17:46:19.422 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (94934 ms total) 2024-02-14T17:46:19.422 INFO:tasks.workunit.client.0.smithi178.stdout: snapshots: [ PASSED ] 11 tests. 2024-02-14T17:46:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:19 smithi184 ceph-mon[33536]: osdmap e163: 8 total, 8 up, 8 in 2024-02-14T17:46:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4026120532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi178-80663-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3873508218' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi178-80767-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:19 smithi184 ceph-mon[33536]: from='client.49027 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi178-80663-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:19 smithi184 ceph-mon[33536]: from='client.49024 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi178-80767-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:19 smithi184 ceph-mon[33536]: pgmap v202: 364 pgs: 300 active+clean, 64 unknown; 592 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:46:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:19.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[30321]: osdmap e163: 8 total, 8 up, 8 in 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4026120532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi178-80663-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3873508218' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi178-80767-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[30321]: from='client.49027 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi178-80663-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[30321]: from='client.49024 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi178-80767-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[30321]: pgmap v202: 364 pgs: 300 active+clean, 64 unknown; 592 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[39811]: osdmap e163: 8 total, 8 up, 8 in 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4026120532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi178-80663-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3873508218' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi178-80767-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[39811]: from='client.49027 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi178-80663-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[39811]: from='client.49024 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi178-80767-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[39811]: pgmap v202: 364 pgs: 300 active+clean, 64 unknown; 592 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:46:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:20 smithi184 ceph-mon[33536]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:20 smithi184 ceph-mon[33536]: from='client.49027 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi178-80663-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:20 smithi184 ceph-mon[33536]: from='client.49024 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi178-80767-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:20 smithi184 ceph-mon[33536]: osdmap e164: 8 total, 8 up, 8 in 2024-02-14T17:46:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3561195577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi178-80647-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3561195577' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi178-80647-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:20.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:20 smithi184 ceph-mon[33536]: osdmap e165: 8 total, 8 up, 8 in 2024-02-14T17:46:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[30321]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[30321]: from='client.49027 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi178-80663-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[30321]: from='client.49024 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi178-80767-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[30321]: osdmap e164: 8 total, 8 up, 8 in 2024-02-14T17:46:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3561195577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi178-80647-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3561195577' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi178-80647-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[30321]: osdmap e165: 8 total, 8 up, 8 in 2024-02-14T17:46:20.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[39811]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:20.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[39811]: from='client.49027 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi178-80663-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:20.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[39811]: from='client.49024 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi178-80767-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:20.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[39811]: osdmap e164: 8 total, 8 up, 8 in 2024-02-14T17:46:20.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3561195577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi178-80647-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:20.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:20.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3561195577' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi178-80647-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:20.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:20 smithi178 ceph-mon[39811]: osdmap e165: 8 total, 8 up, 8 in 2024-02-14T17:46:21.420 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:46:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: pgmap v205: 300 pgs: 1 active, 8 creating+activating, 5 creating+peering, 286 active+clean; 592 KiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:46:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: osdmap e166: 8 total, 8 up, 8 in 2024-02-14T17:46:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi178-80767-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2113902122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi178-80663-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:21.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: from='client.49042 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi178-80663-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:21.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:21 smithi184 ceph-mon[33536]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: pgmap v205: 300 pgs: 1 active, 8 creating+activating, 5 creating+peering, 286 active+clean; 592 KiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: osdmap e166: 8 total, 8 up, 8 in 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi178-80767-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2113902122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi178-80663-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: from='client.49042 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi178-80663-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[39811]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: pgmap v205: 300 pgs: 1 active, 8 creating+activating, 5 creating+peering, 286 active+clean; 592 KiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: osdmap e166: 8 total, 8 up, 8 in 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi178-80767-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2113902122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi178-80663-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: from='client.49042 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi178-80663-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:21.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:21 smithi178 ceph-mon[30321]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:22.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi178-80647-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:22.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:22.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:22 smithi184 ceph-mon[33536]: from='client.49042 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi178-80663-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:23.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi178-80647-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:22 smithi178 ceph-mon[39811]: from='client.49042 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi178-80663-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi178-80647-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:22 smithi178 ceph-mon[30321]: from='client.49042 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi178-80663-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:23.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]': finished 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi178-80647-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: osdmap e167: 8 total, 8 up, 8 in 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi178-80647-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: pgmap v208: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi178-80767-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi178-80767-36"}]': finished 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]': finished 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: osdmap e168: 8 total, 8 up, 8 in 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:23.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:23.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:23 smithi184 ceph-mon[33536]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]': finished 2024-02-14T17:46:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi178-80647-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: osdmap e167: 8 total, 8 up, 8 in 2024-02-14T17:46:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi178-80647-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: pgmap v208: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi178-80767-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi178-80767-36"}]': finished 2024-02-14T17:46:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]': finished 2024-02-14T17:46:24.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: osdmap e168: 8 total, 8 up, 8 in 2024-02-14T17:46:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:24.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[39811]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi178-80984-16"}]': finished 2024-02-14T17:46:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi178-80647-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/541755358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: osdmap e167: 8 total, 8 up, 8 in 2024-02-14T17:46:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi178-80647-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.56522 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]: dispatch 2024-02-14T17:46:24.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: pgmap v208: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:24.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:24.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi178-80767-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi178-80767-36"}]': finished 2024-02-14T17:46:24.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.56522 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi178-80984-16"}]': finished 2024-02-14T17:46:24.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:24.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: osdmap e168: 8 total, 8 up, 8 in 2024-02-14T17:46:24.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:24.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:24.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:23 smithi178 ceph-mon[30321]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi178-80647-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi178-80647-27"}]': finished 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: osdmap e169: 8 total, 8 up, 8 in 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4195036334' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi178-80663-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: from='client.56594 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi178-80663-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:24.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:24 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi178-80647-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi178-80647-27"}]': finished 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: osdmap e169: 8 total, 8 up, 8 in 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4195036334' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi178-80663-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: from='client.56594 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi178-80663-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi178-80647-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi178-80647-27"}]': finished 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: osdmap e169: 8 total, 8 up, 8 in 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4195036334' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi178-80663-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: from='client.56594 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi178-80663-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:25.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:24 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:25.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:25 smithi184 ceph-mon[33536]: pgmap v211: 308 pgs: 48 unknown, 260 active+clean; 592 KiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:25.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:25.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:25 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:25.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:25 smithi184 ceph-mon[33536]: from='client.56594 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi178-80663-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:25.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:25.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:46:25.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:25 smithi184 ceph-mon[33536]: osdmap e170: 8 total, 8 up, 8 in 2024-02-14T17:46:25.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:46:25.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:26.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:46:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[39811]: pgmap v211: 308 pgs: 48 unknown, 260 active+clean; 592 KiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[39811]: from='client.56594 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi178-80663-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:46:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[39811]: osdmap e170: 8 total, 8 up, 8 in 2024-02-14T17:46:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:46:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:26.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[30321]: pgmap v211: 308 pgs: 48 unknown, 260 active+clean; 592 KiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:26.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:26.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:26.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[30321]: from='client.56594 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi178-80663-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:26.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:26.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:46:26.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[30321]: osdmap e170: 8 total, 8 up, 8 in 2024-02-14T17:46:26.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13"}]: dispatch 2024-02-14T17:46:26.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:26 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:26 smithi184 ceph-mon[33536]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]': finished 2024-02-14T17:46:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:26 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13"}]': finished 2024-02-14T17:46:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36"}]': finished 2024-02-14T17:46:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:26 smithi184 ceph-mon[33536]: osdmap e171: 8 total, 8 up, 8 in 2024-02-14T17:46:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:27.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[39811]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]': finished 2024-02-14T17:46:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13"}]': finished 2024-02-14T17:46:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36"}]': finished 2024-02-14T17:46:27.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[39811]: osdmap e171: 8 total, 8 up, 8 in 2024-02-14T17:46:27.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:27.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:27.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:27.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:27.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[30321]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]': finished 2024-02-14T17:46:27.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-13"}]': finished 2024-02-14T17:46:27.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi178-80767-36"}]': finished 2024-02-14T17:46:27.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[30321]: osdmap e171: 8 total, 8 up, 8 in 2024-02-14T17:46:27.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi178-80767-36"}]: dispatch 2024-02-14T17:46:27.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:27.393 INFO:tasks.workunit.client.0.smithi178.stdout:7497 ms total) 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (124 ms) 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (124 ms total) 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (39 ms) 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (1 ms) 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (40 ms total) 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: 2024-02-14T17:46:27.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (27 ms) 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (5 ms) 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (32 ms total) 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (37 ms) 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (6 ms) 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (43 ms total) 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-02-14T17:46:27.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-02-14T17:46:27.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (769 ms) 2024-02-14T17:46:27.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (769 ms total) 2024-02-14T17:46:27.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: 2024-02-14T17:46:27.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-02-14T17:46:27.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (103277 ms total) 2024-02-14T17:46:27.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-02-14T17:46:27.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:27 smithi184 ceph-mon[33536]: pgmap v214: 268 pgs: 8 unknown, 260 active+clean; 585 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 21 op/s 2024-02-14T17:46:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi178-80767-36"}]': finished 2024-02-14T17:46:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi178-80647-27"}]': finished 2024-02-14T17:46:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:27 smithi184 ceph-mon[33536]: osdmap e172: 8 total, 8 up, 8 in 2024-02-14T17:46:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1197740894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi178-80663-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:27 smithi184 ceph-mon[33536]: from='client.56600 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi178-80663-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:28.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[39811]: pgmap v214: 268 pgs: 8 unknown, 260 active+clean; 585 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 21 op/s 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi178-80767-36"}]': finished 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi178-80647-27"}]': finished 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[39811]: osdmap e172: 8 total, 8 up, 8 in 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1197740894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi178-80663-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[39811]: from='client.56600 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi178-80663-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[30321]: pgmap v214: 268 pgs: 8 unknown, 260 active+clean; 585 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 21 op/s 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2555858531' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi178-80767-36"}]': finished 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi178-80647-27"}]': finished 2024-02-14T17:46:28.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[30321]: osdmap e172: 8 total, 8 up, 8 in 2024-02-14T17:46:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi178-80647-27"}]: dispatch 2024-02-14T17:46:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1197740894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi178-80663-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:27 smithi178 ceph-mon[30321]: from='client.56600 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi178-80663-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:28.392 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: Running main() from gmock_main.cc 2024-02-14T17:46:28.392 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-02-14T17:46:28.392 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [----------] Global test environment set-up. 2024-02-14T17:46:28.392 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-02-14T17:46:28.392 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-02-14T17:46:28.392 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.TooBig (8858 ms) 2024-02-14T17:46:28.392 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-02-14T17:46:28.392 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (5744 ms) 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (5497 ms) 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3608 ms) 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (5182 ms) 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (5072 ms) 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (5520 ms) 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3036 ms) 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-14T17:46:28.393 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3026 ms) 2024-02-14T17:46:28.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-02-14T17:46:28.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3768 ms) 2024-02-14T17:46:28.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-02-14T17:46:28.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3001 ms) 2024-02-14T17:46:28.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-02-14T17:46:28.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3020 ms) 2024-02-14T17:46:28.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-02-14T17:46:28.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (2988 ms) 2024-02-14T17:46:28.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-02-14T17:46:28.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2994 ms) 2024-02-14T17:46:28.394 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.Flush (3004 ms) 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3001 ms) 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3011 ms) 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2995 ms) 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3019 ms) 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3265 ms) 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (2769 ms) 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-02-14T17:46:28.395 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3016 ms) 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2936 ms) 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3019 ms) 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3011 ms) 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (2970 ms) 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [----------] 26 tests from LibRadosAio (97331 ms total) 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-02-14T17:46:28.396 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-02-14T17:46:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi178-80647-27"}]': finished 2024-02-14T17:46:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='client.56600 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi178-80663-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: osdmap e173: 8 total, 8 up, 8 in 2024-02-14T17:46:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:28.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:28.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:28.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi178-80647-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:28.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:28 smithi184 ceph-mon[33536]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi178-80647-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi178-80647-27"}]': finished 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='client.56600 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi178-80663-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: osdmap e173: 8 total, 8 up, 8 in 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi178-80647-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[39811]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi178-80647-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2455520277' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi178-80647-27"}]': finished 2024-02-14T17:46:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='client.56600 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi178-80663-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: osdmap e173: 8 total, 8 up, 8 in 2024-02-14T17:46:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:29.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:29.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:29.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:29.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi178-80647-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:29.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:28 smithi178 ceph-mon[30321]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi178-80647-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:29.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:29 smithi184 ceph-mon[33536]: pgmap v217: 300 pgs: 72 unknown, 228 active+clean; 585 KiB data, 557 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:29.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:29.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:29 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:29.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:29 smithi184 ceph-mon[33536]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi178-80647-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:29.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:29 smithi184 ceph-mon[33536]: osdmap e174: 8 total, 8 up, 8 in 2024-02-14T17:46:29.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi178-80647-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:29.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:29 smithi184 ceph-mon[33536]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi178-80647-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:29.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:29 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:46:30.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[30321]: pgmap v217: 300 pgs: 72 unknown, 228 active+clean; 585 KiB data, 557 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[30321]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi178-80647-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[30321]: osdmap e174: 8 total, 8 up, 8 in 2024-02-14T17:46:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi178-80647-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[30321]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi178-80647-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:46:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[39811]: pgmap v217: 300 pgs: 72 unknown, 228 active+clean; 585 KiB data, 557 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:30.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:30.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[39811]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi178-80647-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:30.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[39811]: osdmap e174: 8 total, 8 up, 8 in 2024-02-14T17:46:30.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi178-80647-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:30.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[39811]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi178-80647-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:30.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:29 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:46:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:30 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:46:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:30 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:46:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:30 smithi184 ceph-mon[33536]: pgmap v219: 268 pgs: 40 unknown, 228 active+clean; 585 KiB data, 557 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:30 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:30 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:30 smithi184 ceph-mon[33536]: osdmap e175: 8 total, 8 up, 8 in 2024-02-14T17:46:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/124189124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi178-80663-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:30 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:30 smithi184 ceph-mon[33536]: from='client.49069 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi178-80663-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:46:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:46:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[39811]: pgmap v219: 268 pgs: 40 unknown, 228 active+clean; 585 KiB data, 557 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[39811]: osdmap e175: 8 total, 8 up, 8 in 2024-02-14T17:46:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/124189124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi178-80663-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[39811]: from='client.49069 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi178-80663-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:46:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:46:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[30321]: pgmap v219: 268 pgs: 40 unknown, 228 active+clean; 585 KiB data, 557 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[30321]: osdmap e175: 8 total, 8 up, 8 in 2024-02-14T17:46:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/124189124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi178-80663-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:30 smithi178 ceph-mon[30321]: from='client.49069 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi178-80663-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:31.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:46:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:46:31.714 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:31 smithi184 ceph-mon[33536]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-14T17:46:31.714 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:31.715 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:31 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:31.715 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:31 smithi184 ceph-mon[33536]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi178-80647-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi178-80647-28"}]': finished 2024-02-14T17:46:31.715 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:31 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:31.715 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:31 smithi184 ceph-mon[33536]: from='client.49069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi178-80663-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:31.715 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:31 smithi184 ceph-mon[33536]: osdmap e176: 8 total, 8 up, 8 in 2024-02-14T17:46:31.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:31.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:31 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[39811]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[39811]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi178-80647-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi178-80647-28"}]': finished 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[39811]: from='client.49069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi178-80663-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[39811]: osdmap e176: 8 total, 8 up, 8 in 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[30321]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[30321]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi178-80647-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi178-80647-28"}]': finished 2024-02-14T17:46:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[30321]: from='client.49069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi178-80663-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[30321]: osdmap e176: 8 total, 8 up, 8 in 2024-02-14T17:46:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:31 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:32.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:32 smithi184 ceph-mon[33536]: pgmap v222: 308 pgs: 9 creating+peering, 31 unknown, 268 active+clean; 585 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-02-14T17:46:32.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:32 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-15"}]': finished 2024-02-14T17:46:32.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:32 smithi184 ceph-mon[33536]: osdmap e177: 8 total, 8 up, 8 in 2024-02-14T17:46:32.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-15", "mode": "writeback"}]: dispatch 2024-02-14T17:46:32.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:32 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-15", "mode": "writeback"}]: dispatch 2024-02-14T17:46:33.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[39811]: pgmap v222: 308 pgs: 9 creating+peering, 31 unknown, 268 active+clean; 585 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-02-14T17:46:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-15"}]': finished 2024-02-14T17:46:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[39811]: osdmap e177: 8 total, 8 up, 8 in 2024-02-14T17:46:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-15", "mode": "writeback"}]: dispatch 2024-02-14T17:46:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-15", "mode": "writeback"}]: dispatch 2024-02-14T17:46:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[30321]: pgmap v222: 308 pgs: 9 creating+peering, 31 unknown, 268 active+clean; 585 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-02-14T17:46:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-15"}]': finished 2024-02-14T17:46:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[30321]: osdmap e177: 8 total, 8 up, 8 in 2024-02-14T17:46:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-15", "mode": "writeback"}]: dispatch 2024-02-14T17:46:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:32 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-15", "mode": "writeback"}]: dispatch 2024-02-14T17:46:34.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-15", "mode": "writeback"}]': finished 2024-02-14T17:46:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[39811]: osdmap e178: 8 total, 8 up, 8 in 2024-02-14T17:46:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2370650031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi178-80663-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[39811]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-15", "mode": "writeback"}]': finished 2024-02-14T17:46:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[30321]: osdmap e178: 8 total, 8 up, 8 in 2024-02-14T17:46:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2370650031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi178-80663-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:33 smithi178 ceph-mon[30321]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:34.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:33 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:33 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-15", "mode": "writeback"}]': finished 2024-02-14T17:46:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:33 smithi184 ceph-mon[33536]: osdmap e178: 8 total, 8 up, 8 in 2024-02-14T17:46:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2370650031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi178-80663-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:33 smithi184 ceph-mon[33536]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[39811]: pgmap v225: 300 pgs: 32 unknown, 268 active+clean; 585 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.2 KiB/s wr, 2 op/s 2024-02-14T17:46:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2370650031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi178-80663-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[39811]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]': finished 2024-02-14T17:46:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[39811]: osdmap e179: 8 total, 8 up, 8 in 2024-02-14T17:46:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[39811]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[30321]: pgmap v225: 300 pgs: 32 unknown, 268 active+clean; 585 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.2 KiB/s wr, 2 op/s 2024-02-14T17:46:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2370650031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi178-80663-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[30321]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]': finished 2024-02-14T17:46:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[30321]: osdmap e179: 8 total, 8 up, 8 in 2024-02-14T17:46:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[30321]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:34 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:34 smithi184 ceph-mon[33536]: pgmap v225: 300 pgs: 32 unknown, 268 active+clean; 585 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.2 KiB/s wr, 2 op/s 2024-02-14T17:46:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2370650031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi178-80663-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:34 smithi184 ceph-mon[33536]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi178-80647-28"}]': finished 2024-02-14T17:46:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:34 smithi184 ceph-mon[33536]: osdmap e179: 8 total, 8 up, 8 in 2024-02-14T17:46:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3595865081' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:34 smithi184 ceph-mon[33536]: from='client.49063 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]: dispatch 2024-02-14T17:46:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:34 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:46:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]': finished 2024-02-14T17:46:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: osdmap e180: 8 total, 8 up, 8 in 2024-02-14T17:46:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi178-80647-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[39811]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi178-80647-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]': finished 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: osdmap e180: 8 total, 8 up, 8 in 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi178-80647-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:36.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:35 smithi178 ceph-mon[30321]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi178-80647-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:36.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:36.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: from='client.49063 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi178-80647-28"}]': finished 2024-02-14T17:46:36.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:36.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: osdmap e180: 8 total, 8 up, 8 in 2024-02-14T17:46:36.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:36.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15"}]: dispatch 2024-02-14T17:46:36.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:36.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi178-80647-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:36.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:35 smithi184 ceph-mon[33536]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi178-80647-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[39811]: pgmap v228: 268 pgs: 268 active+clean; 617 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:46:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15"}]': finished 2024-02-14T17:46:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[39811]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi178-80647-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[39811]: osdmap e181: 8 total, 8 up, 8 in 2024-02-14T17:46:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi178-80647-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2259351225' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi178-80663-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[39811]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi178-80647-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:37.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[30321]: pgmap v228: 268 pgs: 268 active+clean; 617 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:46:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15"}]': finished 2024-02-14T17:46:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[30321]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi178-80647-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[30321]: osdmap e181: 8 total, 8 up, 8 in 2024-02-14T17:46:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi178-80647-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2259351225' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi178-80663-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:36 smithi178 ceph-mon[30321]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi178-80647-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:37.133 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:37.133 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:36 smithi184 ceph-mon[33536]: pgmap v228: 268 pgs: 268 active+clean; 617 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:46:37.133 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:36 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:37.133 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-15"}]': finished 2024-02-14T17:46:37.133 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:36 smithi184 ceph-mon[33536]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi178-80647-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:37.133 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:36 smithi184 ceph-mon[33536]: osdmap e181: 8 total, 8 up, 8 in 2024-02-14T17:46:37.133 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi178-80647-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:37.133 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2259351225' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi178-80663-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:37.133 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:36 smithi184 ceph-mon[33536]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi178-80647-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:38.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:37 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:37 smithi184 ceph-mon[33536]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2259351225' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi178-80663-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:37 smithi184 ceph-mon[33536]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:46:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:38.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:37 smithi184 ceph-mon[33536]: osdmap e182: 8 total, 8 up, 8 in 2024-02-14T17:46:38.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[39811]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2259351225' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi178-80663-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[39811]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:46:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[39811]: osdmap e182: 8 total, 8 up, 8 in 2024-02-14T17:46:38.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:38.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:38.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:38.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[30321]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:38.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2259351225' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi178-80663-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:38.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[30321]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:46:38.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:38.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:38.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:38.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:37 smithi178 ceph-mon[30321]: osdmap e182: 8 total, 8 up, 8 in 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (8841 ms) 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (18944 ms) 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (12734 ms) 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2994 ms) 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-02-14T17:46:38.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3788 ms) 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3267 ms) 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3751 ms) 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (2995 ms) 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3009 ms) 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (2994 ms) 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2998 ms) 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3008 ms) 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (2996 ms) 2024-02-14T17:46:38.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3014 ms) 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3023 ms) 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3017 ms) 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3004 ms) 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (2948 ms) 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3014 ms) 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3015 ms) 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (2973 ms) 2024-02-14T17:46:38.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-02-14T17:46:38.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3256 ms) 2024-02-14T17:46:38.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-02-14T17:46:38.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (2752 ms) 2024-02-14T17:46:38.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-02-14T17:46:38.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3013 ms) 2024-02-14T17:46:38.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-02-14T17:46:38.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2984 ms) 2024-02-14T17:46:38.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-02-14T17:46:38.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2998 ms) 2024-02-14T17:46:38.388 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-14T17:46:39.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:38 smithi184 ceph-mon[33536]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:39.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:38 smithi184 ceph-mon[33536]: pgmap v231: 268 pgs: 32 unknown, 236 active+clean; 617 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:46:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:38 smithi184 ceph-mon[33536]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi178-80647-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi178-80647-29"}]': finished 2024-02-14T17:46:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:38 smithi184 ceph-mon[33536]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:46:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:38 smithi184 ceph-mon[33536]: osdmap e183: 8 total, 8 up, 8 in 2024-02-14T17:46:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:38 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:39.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:38 smithi184 ceph-mon[33536]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:39.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[39811]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[39811]: pgmap v231: 268 pgs: 32 unknown, 236 active+clean; 617 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:46:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[39811]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi178-80647-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi178-80647-29"}]': finished 2024-02-14T17:46:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[39811]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:46:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[39811]: osdmap e183: 8 total, 8 up, 8 in 2024-02-14T17:46:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:39.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[39811]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:39.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[30321]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[30321]: pgmap v231: 268 pgs: 32 unknown, 236 active+clean; 617 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:46:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[30321]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi178-80647-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi178-80647-29"}]': finished 2024-02-14T17:46:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[30321]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:46:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[30321]: osdmap e183: 8 total, 8 up, 8 in 2024-02-14T17:46:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:39.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:38 smithi178 ceph-mon[30321]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:46:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:46:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: osdmap e184: 8 total, 8 up, 8 in 2024-02-14T17:46:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1696395209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi178-80663-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.49093 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi178-80663-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: pgmap v234: 300 pgs: 72 unknown, 228 active+clean; 585 KiB data, 559 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:40.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.49093 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi178-80663-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:40.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]': finished 2024-02-14T17:46:40.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:40.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: osdmap e185: 8 total, 8 up, 8 in 2024-02-14T17:46:40.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:40.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:40 smithi184 ceph-mon[33536]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:46:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: osdmap e184: 8 total, 8 up, 8 in 2024-02-14T17:46:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1696395209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi178-80663-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.49093 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi178-80663-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: pgmap v234: 300 pgs: 72 unknown, 228 active+clean; 585 KiB data, 559 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.49093 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi178-80663-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]': finished 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: osdmap e185: 8 total, 8 up, 8 in 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[30321]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:46:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: osdmap e184: 8 total, 8 up, 8 in 2024-02-14T17:46:40.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:40.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:40.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1696395209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi178-80663-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:40.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.49093 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi178-80663-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:40.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:40.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:40.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:40.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: pgmap v234: 300 pgs: 72 unknown, 228 active+clean; 585 KiB data, 559 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:40.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.49093 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi178-80663-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:40.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]': finished 2024-02-14T17:46:40.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:40.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: osdmap e185: 8 total, 8 up, 8 in 2024-02-14T17:46:40.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:40.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:40 smithi178 ceph-mon[39811]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:41.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAinapshotsECPP.SnapRemovePP (2009 ms) 2024-02-14T17:46:41.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-02-14T17:46:41.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2020 ms) 2024-02-14T17:46:41.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-02-14T17:46:41.385 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (1961 ms) 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9000 ms total) 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4003 ms) 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3997 ms) 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4000 ms) 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (12000 ms total) 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-02-14T17:46:41.386 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (117225 ms total) 2024-02-14T17:46:41.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-02-14T17:46:41.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-02-14T17:46:41.387 INFO:tasks.workunit.client.0.smithi178.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-14T17:46:41.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:46:41.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]': finished 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]': finished 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-17"}]': finished 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: osdmap e186: 8 total, 8 up, 8 in 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-17", "mode": "writeback"}]: dispatch 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:41 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-17", "mode": "writeback"}]: dispatch 2024-02-14T17:46:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]': finished 2024-02-14T17:46:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]': finished 2024-02-14T17:46:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-17"}]': finished 2024-02-14T17:46:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: osdmap e186: 8 total, 8 up, 8 in 2024-02-14T17:46:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-17", "mode": "writeback"}]: dispatch 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-17", "mode": "writeback"}]: dispatch 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/134879533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.56588 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]: dispatch 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi178-80647-29"}]': finished 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.56588 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi178-80984-21"}]': finished 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-17"}]': finished 2024-02-14T17:46:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: osdmap e186: 8 total, 8 up, 8 in 2024-02-14T17:46:41.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-17", "mode": "writeback"}]: dispatch 2024-02-14T17:46:41.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1341050149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:41.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.49084 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]: dispatch 2024-02-14T17:46:41.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:41 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-17", "mode": "writeback"}]: dispatch 2024-02-14T17:46:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: pgmap v237: 260 pgs: 260 active+clean; 585 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 6 op/s 2024-02-14T17:46:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi178-80753-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:46:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi178-80753-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:46:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]': finished 2024-02-14T17:46:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-17", "mode": "writeback"}]': finished 2024-02-14T17:46:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi178-80753-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-14T17:46:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: osdmap e187: 8 total, 8 up, 8 in 2024-02-14T17:46:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2673530649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi178-80663-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:42.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: from='client.56642 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi178-80663-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:42.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:42.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: pgmap v237: 260 pgs: 260 active+clean; 585 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 6 op/s 2024-02-14T17:46:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi178-80753-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:46:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi178-80753-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:46:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]': finished 2024-02-14T17:46:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-17", "mode": "writeback"}]': finished 2024-02-14T17:46:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi178-80753-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-14T17:46:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: osdmap e187: 8 total, 8 up, 8 in 2024-02-14T17:46:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2673530649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi178-80663-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: from='client.56642 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi178-80663-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: pgmap v237: 260 pgs: 260 active+clean; 585 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 6 op/s 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi178-80753-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi178-80753-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: from='client.49084 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi178-80647-29"}]': finished 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-17", "mode": "writeback"}]': finished 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi178-80753-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: osdmap e187: 8 total, 8 up, 8 in 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2673530649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi178-80663-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: from='client.56642 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi178-80663-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi178-80647-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi178-80647-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.56642 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi178-80663-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi178-80647-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: osdmap e188: 8 total, 8 up, 8 in 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi178-80647-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi178-80647-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:43.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:43.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:43.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:43 smithi184 ceph-mon[33536]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi178-80647-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi178-80647-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.56642 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi178-80663-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi178-80647-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: osdmap e188: 8 total, 8 up, 8 in 2024-02-14T17:46:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi178-80647-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi178-80647-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:43.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:43.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi178-80647-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:43.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi178-80647-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:43.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:43.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:43.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.56642 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi178-80663-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi178-80647-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: osdmap e188: 8 total, 8 up, 8 in 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi178-80647-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi178-80647-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17"}]: dispatch 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:43.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[39811]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:43.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:43.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:43.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:43 smithi178 ceph-mon[30321]: from='client.34697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:46:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:44 smithi184 ceph-mon[33536]: pgmap v240: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-14T17:46:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:44 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:44.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:44 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17"}]': finished 2024-02-14T17:46:45.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:44 smithi178 ceph-mon[30321]: pgmap v240: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-14T17:46:45.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:44 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:44 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17"}]': finished 2024-02-14T17:46:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:44 smithi178 ceph-mon[39811]: pgmap v240: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-14T17:46:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:44 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:44 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-17"}]': finished 2024-02-14T17:46:45.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:45 smithi184 ceph-mon[33536]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:46:45.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:45 smithi184 ceph-mon[33536]: osdmap e189: 8 total, 8 up, 8 in 2024-02-14T17:46:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:45 smithi184 ceph-mon[33536]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi178-80647-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi178-80647-30"}]': finished 2024-02-14T17:46:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:45 smithi184 ceph-mon[33536]: osdmap e190: 8 total, 8 up, 8 in 2024-02-14T17:46:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/727265652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi178-80663-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:45.984 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:46:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:45.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[39811]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:46:45.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:45.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[39811]: osdmap e189: 8 total, 8 up, 8 in 2024-02-14T17:46:45.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[39811]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi178-80647-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi178-80647-30"}]': finished 2024-02-14T17:46:45.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[39811]: osdmap e190: 8 total, 8 up, 8 in 2024-02-14T17:46:45.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/727265652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi178-80663-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[30321]: from='client.34697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi178-80753-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:46:45.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4117746465' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:46:45.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[30321]: osdmap e189: 8 total, 8 up, 8 in 2024-02-14T17:46:45.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[30321]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi178-80647-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi178-80647-30"}]': finished 2024-02-14T17:46:45.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[30321]: osdmap e190: 8 total, 8 up, 8 in 2024-02-14T17:46:45.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/727265652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi178-80663-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:46.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.56654 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi178-80663-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi178-80753-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi178-80753-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: pgmap v243: 236 pgs: 29 creating+peering, 11 unknown, 196 active+clean; 583 KiB data, 532 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.56654 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi178-80663-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi178-80753-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: osdmap e191: 8 total, 8 up, 8 in 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi178-80753-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:46 smithi184 ceph-mon[33536]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi178-80753-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.56654 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi178-80663-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi178-80753-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi178-80753-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: pgmap v243: 236 pgs: 29 creating+peering, 11 unknown, 196 active+clean; 583 KiB data, 532 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.56654 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi178-80663-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi178-80753-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: osdmap e191: 8 total, 8 up, 8 in 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi178-80753-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[39811]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi178-80753-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.56654 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi178-80663-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi178-80753-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi178-80753-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: pgmap v243: 236 pgs: 29 creating+peering, 11 unknown, 196 active+clean; 583 KiB data, 532 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.56654 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi178-80663-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi178-80753-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: osdmap e191: 8 total, 8 up, 8 in 2024-02-14T17:46:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:47.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi178-80753-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:47.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:47.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:46 smithi178 ceph-mon[30321]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi178-80753-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:48.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:48.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:48.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:48 smithi184 ceph-mon[33536]: osdmap e192: 8 total, 8 up, 8 in 2024-02-14T17:46:48.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:48.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:48.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:48 smithi184 ceph-mon[33536]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:48.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:48 smithi184 ceph-mon[33536]: pgmap v246: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 532 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:48.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[30321]: osdmap e192: 8 total, 8 up, 8 in 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[30321]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[30321]: pgmap v246: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 532 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[39811]: osdmap e192: 8 total, 8 up, 8 in 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[39811]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:48 smithi178 ceph-mon[39811]: pgmap v246: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 532 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: expected=4 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:cfc208b3:::3:head 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:cfc208b3:::3:head expected=12:cfc208b3:::3:head 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:cfc208b3:::3:head -> 3 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=3 expected=3 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:863748b0:::15:head 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:863748b0:::15:head expected=12:863748b0:::15:head 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:863748b0:::15:head -> 15 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=15 expected=15 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:6cac518f:::0:head 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:6cac518f:::0:head expected=12:6cac518f:::0:head 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:6cac518f:::0:head -> 0 2024-02-14T17:46:49.474 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=0 expected=0 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:62a1935d:::14:head 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:62a1935d:::14:head expected=12:62a1935d:::14:head 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:62a1935d:::14:head -> 14 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=14 expected=14 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:c4fdafeb:::6:head 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:c4fdafeb:::6:head expected=12:c4fdafeb:::6:head 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:c4fdafeb:::6:head -> 6 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=6 expected=6 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:5c6b0b28:::7:head 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:5c6b0b28:::7:head expected=12:5c6b0b28:::7:head 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:5c6b0b28:::7:head -> 7 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=7 expected=7 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:566253c9:::13:head 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:566253c9:::13:head expected=12:566253c9:::13:head 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:566253c9:::13:head -> 13 2024-02-14T17:46:49.475 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=13 expected=13 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : seek to 12:52ea6a34:::10:head 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : cursor()=12:52ea6a34:::10:head expected=12:52ea6a34:::10:head 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: > 12:52ea6a34:::10:head -> 10 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: : entry=10 expected=10 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (237 ms) 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (104769 ms) 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: split 0/5 -> MIN 12:33333333::::head 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: split 1/5 -> 12:33333333::::head 12:66666666::::head 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: split 2/5 -> 12:66666666::::head 12:99999999::::head 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: split 3/5 -> 12:99999999::::head 12:cccccccc::::head 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: split 4/5 -> 12:cccccccc::::head MAX 2024-02-14T17:46:49.476 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (6240 ms) 2024-02-14T17:46:49.477 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [----------] 7 tests from LibRadosList (112817 ms total) 2024-02-14T17:46:49.477 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 2024-02-14T17:46:49.477 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-02-14T17:46:49.477 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-02-14T17:46:49.477 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1039 ms) 2024-02-14T17:46:49.477 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-02-14T17:46:49.477 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: myset foo1,foo2,foo3 2024-02-14T17:46:49.477 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo1 2024-02-14T17:46:49.477 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo2 2024-02-14T17:46:49.477 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo3 2024-02-14T17:46:49.477 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: myset foo1,foo4,foo5 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo4 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo5 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo1 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: myset foo6,foo7 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo7 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: foo6 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: ns1:foo4 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: ns1:foo5 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: ns2:foo7 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: ns2:foo6 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: ns1:foo1 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: :foo1 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: :foo2 2024-02-14T17:46:49.478 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: :foo3 2024-02-14T17:46:49.479 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (31 ms) 2024-02-14T17:46:49.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:49 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:49.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:49 smithi184 ceph-mon[33536]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi178-80753-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi178-80753-2"}]': finished 2024-02-14T17:46:49.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:49 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:49.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:49 smithi184 ceph-mon[33536]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]': finished 2024-02-14T17:46:49.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:49 smithi184 ceph-mon[33536]: osdmap e193: 8 total, 8 up, 8 in 2024-02-14T17:46:49.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:49.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:49.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2409262595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi178-80663-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:49.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:49 smithi184 ceph-mon[33536]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:49.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:49 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:49.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:49 smithi184 ceph-mon[33536]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi178-80663-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:49.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[39811]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi178-80753-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi178-80753-2"}]': finished 2024-02-14T17:46:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[39811]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]': finished 2024-02-14T17:46:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[39811]: osdmap e193: 8 total, 8 up, 8 in 2024-02-14T17:46:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2409262595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi178-80663-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[39811]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[39811]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi178-80663-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[30321]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi178-80753-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi178-80753-2"}]': finished 2024-02-14T17:46:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[30321]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi178-80647-30"}]': finished 2024-02-14T17:46:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[30321]: osdmap e193: 8 total, 8 up, 8 in 2024-02-14T17:46:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/559661002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2409262595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi178-80663-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[30321]: from='client.56648 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]: dispatch 2024-02-14T17:46:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:49.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:49 smithi178 ceph-mon[30321]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi178-80663-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:50.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]': finished 2024-02-14T17:46:50.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-19"}]': finished 2024-02-14T17:46:50.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi178-80663-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:50.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: osdmap e194: 8 total, 8 up, 8 in 2024-02-14T17:46:50.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-19", "mode": "writeback"}]: dispatch 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-19", "mode": "writeback"}]: dispatch 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi178-80647-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi178-80647-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2409262595' entity='client.admin' cmd=[{ 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: "prefix": "osd pool set", 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: "pool": "PoolEIOFlag_smithi178-80663-33", 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: "var": "eio", 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: "val": "true" 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: }]: dispatch 2024-02-14T17:46:50.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: from='client.56666 ' entity='client.admin' cmd=[{ 2024-02-14T17:46:50.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: "prefix": "osd pool set", 2024-02-14T17:46:50.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: "pool": "PoolEIOFlag_smithi178-80663-33", 2024-02-14T17:46:50.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: "var": "eio", 2024-02-14T17:46:50.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: "val": "true" 2024-02-14T17:46:50.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: }]: dispatch 2024-02-14T17:46:50.722 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:50 smithi184 ceph-mon[33536]: pgmap v249: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 532 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]': finished 2024-02-14T17:46:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-19"}]': finished 2024-02-14T17:46:50.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi178-80663-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:50.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: osdmap e194: 8 total, 8 up, 8 in 2024-02-14T17:46:50.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-19", "mode": "writeback"}]: dispatch 2024-02-14T17:46:50.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-19", "mode": "writeback"}]: dispatch 2024-02-14T17:46:50.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi178-80647-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:50.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi178-80647-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:50.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2409262595' entity='client.admin' cmd=[{ 2024-02-14T17:46:50.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: "prefix": "osd pool set", 2024-02-14T17:46:50.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: "pool": "PoolEIOFlag_smithi178-80663-33", 2024-02-14T17:46:50.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: "var": "eio", 2024-02-14T17:46:50.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: "val": "true" 2024-02-14T17:46:50.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: }]: dispatch 2024-02-14T17:46:50.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: from='client.56666 ' entity='client.admin' cmd=[{ 2024-02-14T17:46:50.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: "prefix": "osd pool set", 2024-02-14T17:46:50.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: "pool": "PoolEIOFlag_smithi178-80663-33", 2024-02-14T17:46:50.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: "var": "eio", 2024-02-14T17:46:50.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: "val": "true" 2024-02-14T17:46:50.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: }]: dispatch 2024-02-14T17:46:50.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[30321]: pgmap v249: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 532 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:50.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.56648 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi178-80647-30"}]': finished 2024-02-14T17:46:50.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-19"}]': finished 2024-02-14T17:46:50.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi178-80663-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:50.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: osdmap e194: 8 total, 8 up, 8 in 2024-02-14T17:46:50.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-19", "mode": "writeback"}]: dispatch 2024-02-14T17:46:50.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-19", "mode": "writeback"}]: dispatch 2024-02-14T17:46:50.838 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.839 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.839 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.839 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:50.839 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi178-80647-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:50.839 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi178-80647-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:50.839 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2409262595' entity='client.admin' cmd=[{ 2024-02-14T17:46:50.839 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: "prefix": "osd pool set", 2024-02-14T17:46:50.839 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: "pool": "PoolEIOFlag_smithi178-80663-33", 2024-02-14T17:46:50.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: "var": "eio", 2024-02-14T17:46:50.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: "val": "true" 2024-02-14T17:46:50.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: }]: dispatch 2024-02-14T17:46:50.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: from='client.56666 ' entity='client.admin' cmd=[{ 2024-02-14T17:46:50.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: "prefix": "osd pool set", 2024-02-14T17:46:50.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: "pool": "PoolEIOFlag_smithi178-80663-33", 2024-02-14T17:46:50.840 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: "var": "eio", 2024-02-14T17:46:50.841 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: "val": "true" 2024-02-14T17:46:50.841 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: }]: dispatch 2024-02-14T17:46:50.841 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:50 smithi178 ceph-mon[39811]: pgmap v249: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 532 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:46:51.405 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:46:51.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-19", "mode": "writeback"}]': finished 2024-02-14T17:46:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi178-80647-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: from='client.56666 ' entity='client.admin' cmd='[{ 2024-02-14T17:46:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: "prefix": "osd pool set", 2024-02-14T17:46:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: "pool": "PoolEIOFlag_smithi178-80663-33", 2024-02-14T17:46:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: "var": "eio", 2024-02-14T17:46:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: "val": "true" 2024-02-14T17:46:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: }]': finished 2024-02-14T17:46:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: osdmap e195: 8 total, 8 up, 8 in 2024-02-14T17:46:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi178-80647-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi178-80647-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:51.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:51.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:51.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:51.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:51.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:51 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-19", "mode": "writeback"}]': finished 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi178-80647-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: from='client.56666 ' entity='client.admin' cmd='[{ 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: "prefix": "osd pool set", 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: "pool": "PoolEIOFlag_smithi178-80663-33", 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: "var": "eio", 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: "val": "true" 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: }]': finished 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: osdmap e195: 8 total, 8 up, 8 in 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi178-80647-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi178-80647-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-19", "mode": "writeback"}]': finished 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi178-80647-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: from='client.56666 ' entity='client.admin' cmd='[{ 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: "prefix": "osd pool set", 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: "pool": "PoolEIOFlag_smithi178-80663-33", 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: "var": "eio", 2024-02-14T17:46:51.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: "val": "true" 2024-02-14T17:46:51.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: }]': finished 2024-02-14T17:46:51.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: osdmap e195: 8 total, 8 up, 8 in 2024-02-14T17:46:51.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi178-80647-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:51.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi178-80647-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:51.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:51.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:51.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:51.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:51.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:51 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:52.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:52 smithi184 ceph-mon[33536]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]': finished 2024-02-14T17:46:52.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:52 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:52.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:52.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:52.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:52 smithi184 ceph-mon[33536]: osdmap e196: 8 total, 8 up, 8 in 2024-02-14T17:46:52.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:52 smithi184 ceph-mon[33536]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:52.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:52 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:52.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:52 smithi184 ceph-mon[33536]: pgmap v252: 228 pgs: 228 active+clean; 583 KiB data, 540 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:46:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[30321]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]': finished 2024-02-14T17:46:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[30321]: osdmap e196: 8 total, 8 up, 8 in 2024-02-14T17:46:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[30321]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[30321]: pgmap v252: 228 pgs: 228 active+clean; 583 KiB data, 540 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:46:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[39811]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi178-80753-2"}]': finished 2024-02-14T17:46:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:46:52.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3448716196' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:52.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:52.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[39811]: osdmap e196: 8 total, 8 up, 8 in 2024-02-14T17:46:52.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[39811]: from='client.49117 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]: dispatch 2024-02-14T17:46:52.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19"}]: dispatch 2024-02-14T17:46:52.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:52 smithi178 ceph-mon[39811]: pgmap v252: 228 pgs: 228 active+clean; 583 KiB data, 540 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:46:53.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:53 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:53.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:53 smithi184 ceph-mon[33536]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi178-80647-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi178-80647-31"}]': finished 2024-02-14T17:46:53.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:53 smithi184 ceph-mon[33536]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]': finished 2024-02-14T17:46:53.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:53 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19"}]': finished 2024-02-14T17:46:53.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:53 smithi184 ceph-mon[33536]: osdmap e197: 8 total, 8 up, 8 in 2024-02-14T17:46:53.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:53 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3274741109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi178-80663-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:53.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:53 smithi184 ceph-mon[33536]: from='client.56678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi178-80663-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:53.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:53.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[39811]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi178-80647-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi178-80647-31"}]': finished 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[39811]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]': finished 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19"}]': finished 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[39811]: osdmap e197: 8 total, 8 up, 8 in 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3274741109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi178-80663-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[39811]: from='client.56678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi178-80663-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[30321]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi178-80647-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi178-80647-31"}]': finished 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[30321]: from='client.49117 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi178-80753-2"}]': finished 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-19"}]': finished 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[30321]: osdmap e197: 8 total, 8 up, 8 in 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3274741109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi178-80663-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:53 smithi178 ceph-mon[30321]: from='client.56678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi178-80663-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:54 smithi184 ceph-mon[33536]: from='client.56678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi178-80663-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:54 smithi184 ceph-mon[33536]: osdmap e198: 8 total, 8 up, 8 in 2024-02-14T17:46:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4012098947' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80753-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:54 smithi184 ceph-mon[33536]: from='client.56684 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80753-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:54 smithi184 ceph-mon[33536]: pgmap v255: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 540 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:54 smithi184 ceph-mon[33536]: from='client.56684 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80753-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:54 smithi184 ceph-mon[33536]: osdmap e199: 8 total, 8 up, 8 in 2024-02-14T17:46:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:54 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:54 smithi184 ceph-mon[33536]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[30321]: from='client.56678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi178-80663-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[30321]: osdmap e198: 8 total, 8 up, 8 in 2024-02-14T17:46:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4012098947' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80753-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[30321]: from='client.56684 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80753-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[30321]: pgmap v255: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 540 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[30321]: from='client.56684 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80753-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[30321]: osdmap e199: 8 total, 8 up, 8 in 2024-02-14T17:46:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[30321]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[39811]: from='client.56678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi178-80663-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[39811]: osdmap e198: 8 total, 8 up, 8 in 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4012098947' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80753-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[39811]: from='client.56684 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80753-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[39811]: pgmap v255: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 540 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[39811]: from='client.56684 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80753-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[39811]: osdmap e199: 8 total, 8 up, 8 in 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:54.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:54 smithi178 ceph-mon[39811]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:55.435 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 1 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 10 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 13 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 7 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 14 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 0 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 15 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 11 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 5 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 8 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 6 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 3 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 4 0 2024-02-14T17:46:55.436 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 12 0 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 9 0 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 2 0 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (43 ms) 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [----------] 3 tests from LibRadosListEC (1113 ms total) 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3004 ms) 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [----------] 1 test from LibRadosListNP (3004 ms total) 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [----------] Global test environment tear-down 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [==========] 11 tests from 3 test suites ran. (131328 ms total) 2024-02-14T17:46:55.437 INFO:tasks.workunit.client.0.smithi178.stdout: api_list: [ PASSED ] 11 tests. 2024-02-14T17:46:55.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:55 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:55.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4012098947' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi178-80753-3","pool2":"test-rados-api-smithi178-80753-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:46:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:55 smithi184 ceph-mon[33536]: from='client.56684 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi178-80753-3","pool2":"test-rados-api-smithi178-80753-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:46:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:55 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:55 smithi184 ceph-mon[33536]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]': finished 2024-02-14T17:46:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:55 smithi184 ceph-mon[33536]: from='client.56684 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi178-80753-3","pool2":"test-rados-api-smithi178-80753-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-14T17:46:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:55 smithi184 ceph-mon[33536]: osdmap e200: 8 total, 8 up, 8 in 2024-02-14T17:46:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2329670108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi178-80663-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:55 smithi184 ceph-mon[33536]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:55 smithi184 ceph-mon[33536]: from='client.49147 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi178-80663-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:55.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:46:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4012098947' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi178-80753-3","pool2":"test-rados-api-smithi178-80753-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:46:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[39811]: from='client.56684 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi178-80753-3","pool2":"test-rados-api-smithi178-80753-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:46:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[39811]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]': finished 2024-02-14T17:46:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[39811]: from='client.56684 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi178-80753-3","pool2":"test-rados-api-smithi178-80753-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-14T17:46:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[39811]: osdmap e200: 8 total, 8 up, 8 in 2024-02-14T17:46:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2329670108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi178-80663-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[39811]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[39811]: from='client.49147 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi178-80663-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4012098947' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi178-80753-3","pool2":"test-rados-api-smithi178-80753-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[30321]: from='client.56684 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi178-80753-3","pool2":"test-rados-api-smithi178-80753-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[30321]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi178-80647-31"}]': finished 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[30321]: from='client.56684 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi178-80753-3","pool2":"test-rados-api-smithi178-80753-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[30321]: osdmap e200: 8 total, 8 up, 8 in 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1349426198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2329670108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi178-80663-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[30321]: from='client.56672 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]: dispatch 2024-02-14T17:46:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:55 smithi178 ceph-mon[30321]: from='client.49147 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi178-80663-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:56 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:56 smithi184 ceph-mon[33536]: pgmap v258: 260 pgs: 32 creating+peering, 30 unknown, 198 active+clean; 583 KiB data, 525 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:46:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:56 smithi184 ceph-mon[33536]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]': finished 2024-02-14T17:46:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:56 smithi184 ceph-mon[33536]: from='client.49147 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi178-80663-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:56 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:56 smithi184 ceph-mon[33536]: osdmap e201: 8 total, 8 up, 8 in 2024-02-14T17:46:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:46:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:56 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:46:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[39811]: pgmap v258: 260 pgs: 32 creating+peering, 30 unknown, 198 active+clean; 583 KiB data, 525 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:46:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[39811]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]': finished 2024-02-14T17:46:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[39811]: from='client.49147 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi178-80663-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[39811]: osdmap e201: 8 total, 8 up, 8 in 2024-02-14T17:46:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:46:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:46:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:46:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[30321]: pgmap v258: 260 pgs: 32 creating+peering, 30 unknown, 198 active+clean; 583 KiB data, 525 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:46:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[30321]: from='client.56672 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi178-80647-31"}]': finished 2024-02-14T17:46:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[30321]: from='client.49147 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi178-80663-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:46:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:46:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[30321]: osdmap e201: 8 total, 8 up, 8 in 2024-02-14T17:46:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:46:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:56 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:46:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi178-80647-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi178-80647-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-21"}]': finished 2024-02-14T17:46:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi178-80647-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: osdmap e202: 8 total, 8 up, 8 in 2024-02-14T17:46:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-21", "mode": "writeback"}]: dispatch 2024-02-14T17:46:57.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi178-80647-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:57.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-21", "mode": "writeback"}]: dispatch 2024-02-14T17:46:57.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:57 smithi184 ceph-mon[33536]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi178-80647-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi178-80647-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi178-80647-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-21"}]': finished 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi178-80647-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: osdmap e202: 8 total, 8 up, 8 in 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-21", "mode": "writeback"}]: dispatch 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi178-80647-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-21", "mode": "writeback"}]: dispatch 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[39811]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi178-80647-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi178-80647-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi178-80647-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-21"}]': finished 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi178-80647-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: osdmap e202: 8 total, 8 up, 8 in 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-21", "mode": "writeback"}]: dispatch 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi178-80647-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-21", "mode": "writeback"}]: dispatch 2024-02-14T17:46:58.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:57 smithi178 ceph-mon[30321]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi178-80647-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:46:59.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[39811]: pgmap v261: 228 pgs: 30 creating+peering, 198 active+clean; 583 KiB data, 525 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:46:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-21", "mode": "writeback"}]': finished 2024-02-14T17:46:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[39811]: osdmap e203: 8 total, 8 up, 8 in 2024-02-14T17:46:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1829663194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[39811]: from='client.56699 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[30321]: pgmap v261: 228 pgs: 30 creating+peering, 198 active+clean; 583 KiB data, 525 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:46:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-21", "mode": "writeback"}]': finished 2024-02-14T17:46:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[30321]: osdmap e203: 8 total, 8 up, 8 in 2024-02-14T17:46:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1829663194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:46:58 smithi178 ceph-mon[30321]: from='client.56699 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:59.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:58 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:46:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:58 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:58 smithi184 ceph-mon[33536]: pgmap v261: 228 pgs: 30 creating+peering, 198 active+clean; 583 KiB data, 525 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:46:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:58 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:46:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:58 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-21", "mode": "writeback"}]': finished 2024-02-14T17:46:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:58 smithi184 ceph-mon[33536]: osdmap e203: 8 total, 8 up, 8 in 2024-02-14T17:46:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:58 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:46:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1829663194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:46:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:46:58 smithi184 ceph-mon[33536]: from='client.56699 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:00 smithi184 ceph-mon[33536]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi178-80647-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi178-80647-32"}]': finished 2024-02-14T17:47:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:00 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:00 smithi184 ceph-mon[33536]: from='client.56699 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:00 smithi184 ceph-mon[33536]: osdmap e204: 8 total, 8 up, 8 in 2024-02-14T17:47:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:47:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:00 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:47:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:00 smithi184 ceph-mon[33536]: pgmap v264: 268 pgs: 40 unknown, 30 creating+peering, 198 active+clean; 583 KiB data, 525 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:00 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[39811]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi178-80647-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi178-80647-32"}]': finished 2024-02-14T17:47:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[39811]: from='client.56699 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[39811]: osdmap e204: 8 total, 8 up, 8 in 2024-02-14T17:47:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:47:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:47:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[39811]: pgmap v264: 268 pgs: 40 unknown, 30 creating+peering, 198 active+clean; 583 KiB data, 525 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[30321]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi178-80647-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi178-80647-32"}]': finished 2024-02-14T17:47:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[30321]: from='client.56699 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[30321]: osdmap e204: 8 total, 8 up, 8 in 2024-02-14T17:47:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:47:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21"}]: dispatch 2024-02-14T17:47:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[30321]: pgmap v264: 268 pgs: 40 unknown, 30 creating+peering, 198 active+clean; 583 KiB data, 525 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:00 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:01.453 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:47:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:47:01.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:01 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:01 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21"}]': finished 2024-02-14T17:47:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:01 smithi184 ceph-mon[33536]: osdmap e205: 8 total, 8 up, 8 in 2024-02-14T17:47:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/466536325' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:01 smithi184 ceph-mon[33536]: from='client.56705 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:01 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:01.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21"}]': finished 2024-02-14T17:47:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[30321]: osdmap e205: 8 total, 8 up, 8 in 2024-02-14T17:47:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/466536325' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[30321]: from='client.56705 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-21"}]': finished 2024-02-14T17:47:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[39811]: osdmap e205: 8 total, 8 up, 8 in 2024-02-14T17:47:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/466536325' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[39811]: from='client.56705 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:01.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:01 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:02 smithi184 ceph-mon[33536]: from='client.56705 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:02 smithi184 ceph-mon[33536]: osdmap e206: 8 total, 8 up, 8 in 2024-02-14T17:47:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:02 smithi184 ceph-mon[33536]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:02 smithi184 ceph-mon[33536]: pgmap v267: 260 pgs: 3 creating+peering, 29 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:02 smithi178 ceph-mon[39811]: from='client.56705 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:02 smithi178 ceph-mon[39811]: osdmap e206: 8 total, 8 up, 8 in 2024-02-14T17:47:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:02 smithi178 ceph-mon[39811]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:02 smithi178 ceph-mon[39811]: pgmap v267: 260 pgs: 3 creating+peering, 29 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:02 smithi178 ceph-mon[30321]: from='client.56705 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:02 smithi178 ceph-mon[30321]: osdmap e206: 8 total, 8 up, 8 in 2024-02-14T17:47:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:02 smithi178 ceph-mon[30321]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:02 smithi178 ceph-mon[30321]: pgmap v267: 260 pgs: 3 creating+peering, 29 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[39811]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]': finished 2024-02-14T17:47:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[39811]: osdmap e207: 8 total, 8 up, 8 in 2024-02-14T17:47:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[39811]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4133703422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[39811]: from='client.56711 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[30321]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]': finished 2024-02-14T17:47:03.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[30321]: osdmap e207: 8 total, 8 up, 8 in 2024-02-14T17:47:03.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:03.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[30321]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:03.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4133703422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:03.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:03.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[30321]: from='client.56711 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:03.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:03 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:03 smithi184 ceph-mon[33536]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi178-80647-32"}]': finished 2024-02-14T17:47:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:03 smithi184 ceph-mon[33536]: osdmap e207: 8 total, 8 up, 8 in 2024-02-14T17:47:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4113155950' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:03.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:03 smithi184 ceph-mon[33536]: from='client.56693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]: dispatch 2024-02-14T17:47:03.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4133703422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:03.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:03.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:03 smithi184 ceph-mon[33536]: from='client.56711 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:03.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:03 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[30321]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]': finished 2024-02-14T17:47:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[30321]: from='client.56711 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[30321]: osdmap e208: 8 total, 8 up, 8 in 2024-02-14T17:47:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[30321]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[30321]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi178-80647-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:04.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[30321]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi178-80647-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:04.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[30321]: pgmap v270: 324 pgs: 3 creating+peering, 93 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:04.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[39811]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]': finished 2024-02-14T17:47:04.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[39811]: from='client.56711 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:04.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:04.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[39811]: osdmap e208: 8 total, 8 up, 8 in 2024-02-14T17:47:04.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[39811]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[39811]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi178-80647-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:04.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[39811]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi178-80647-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:04.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:04 smithi178 ceph-mon[39811]: pgmap v270: 324 pgs: 3 creating+peering, 93 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:04.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:04 smithi184 ceph-mon[33536]: from='client.56693 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi178-80647-32"}]': finished 2024-02-14T17:47:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:04 smithi184 ceph-mon[33536]: from='client.56711 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi178-80663-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:04 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:04 smithi184 ceph-mon[33536]: osdmap e208: 8 total, 8 up, 8 in 2024-02-14T17:47:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:04 smithi184 ceph-mon[33536]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:04 smithi184 ceph-mon[33536]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi178-80647-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:04 smithi184 ceph-mon[33536]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi178-80647-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:04 smithi184 ceph-mon[33536]: pgmap v270: 324 pgs: 3 creating+peering, 93 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:05.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:47:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[39811]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi178-80647-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[39811]: osdmap e209: 8 total, 8 up, 8 in 2024-02-14T17:47:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi178-80647-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[39811]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi178-80647-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[30321]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi178-80647-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[30321]: osdmap e209: 8 total, 8 up, 8 in 2024-02-14T17:47:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi178-80647-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[30321]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi178-80647-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:05 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:05.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:05 smithi184 ceph-mon[33536]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi178-80647-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:05 smithi184 ceph-mon[33536]: osdmap e209: 8 total, 8 up, 8 in 2024-02-14T17:47:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi178-80647-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:05 smithi184 ceph-mon[33536]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi178-80647-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:05 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:06.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:06 smithi178 ceph-mon[39811]: osdmap e210: 8 total, 8 up, 8 in 2024-02-14T17:47:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:06 smithi178 ceph-mon[39811]: pgmap v273: 260 pgs: 260 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:47:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:06 smithi178 ceph-mon[30321]: osdmap e210: 8 total, 8 up, 8 in 2024-02-14T17:47:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:06 smithi178 ceph-mon[30321]: pgmap v273: 260 pgs: 260 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:47:06.870 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:06.870 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:06.870 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:06 smithi184 ceph-mon[33536]: osdmap e210: 8 total, 8 up, 8 in 2024-02-14T17:47:06.870 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:06.871 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:06 smithi184 ceph-mon[33536]: pgmap v273: 260 pgs: 260 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:47:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:07 smithi178 ceph-mon[39811]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi178-80647-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi178-80647-33"}]': finished 2024-02-14T17:47:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:07 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-23"}]': finished 2024-02-14T17:47:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:07 smithi178 ceph-mon[39811]: osdmap e211: 8 total, 8 up, 8 in 2024-02-14T17:47:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-23", "mode": "writeback"}]: dispatch 2024-02-14T17:47:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:07 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-23", "mode": "writeback"}]: dispatch 2024-02-14T17:47:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:07 smithi178 ceph-mon[30321]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi178-80647-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi178-80647-33"}]': finished 2024-02-14T17:47:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:07 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-23"}]': finished 2024-02-14T17:47:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:07 smithi178 ceph-mon[30321]: osdmap e211: 8 total, 8 up, 8 in 2024-02-14T17:47:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-23", "mode": "writeback"}]: dispatch 2024-02-14T17:47:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:07 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-23", "mode": "writeback"}]: dispatch 2024-02-14T17:47:07.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:07 smithi184 ceph-mon[33536]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi178-80647-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi178-80647-33"}]': finished 2024-02-14T17:47:07.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:07 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-23"}]': finished 2024-02-14T17:47:07.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:07 smithi184 ceph-mon[33536]: osdmap e211: 8 total, 8 up, 8 in 2024-02-14T17:47:07.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-23", "mode": "writeback"}]: dispatch 2024-02-14T17:47:07.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:07 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-23", "mode": "writeback"}]: dispatch 2024-02-14T17:47:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-23", "mode": "writeback"}]': finished 2024-02-14T17:47:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[30321]: osdmap e212: 8 total, 8 up, 8 in 2024-02-14T17:47:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/917260148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[30321]: from='client.56723 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[30321]: pgmap v276: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:47:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-23", "mode": "writeback"}]': finished 2024-02-14T17:47:08.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[39811]: osdmap e212: 8 total, 8 up, 8 in 2024-02-14T17:47:08.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/917260148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:08.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[39811]: from='client.56723 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:08.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:08 smithi178 ceph-mon[39811]: pgmap v276: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:47:08.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:08 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:08.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:08 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-23", "mode": "writeback"}]': finished 2024-02-14T17:47:08.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:08 smithi184 ceph-mon[33536]: osdmap e212: 8 total, 8 up, 8 in 2024-02-14T17:47:08.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/917260148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:08.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:08 smithi184 ceph-mon[33536]: from='client.56723 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:08.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:08 smithi184 ceph-mon[33536]: pgmap v276: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:47:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[30321]: from='client.56723 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[30321]: osdmap e213: 8 total, 8 up, 8 in 2024-02-14T17:47:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[30321]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:09.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[39811]: from='client.56723 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:09.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[39811]: osdmap e213: 8 total, 8 up, 8 in 2024-02-14T17:47:09.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:09.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[39811]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:09.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:09.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:09 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:09.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:09 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:09.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:09 smithi184 ceph-mon[33536]: from='client.56723 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:09.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:09 smithi184 ceph-mon[33536]: osdmap e213: 8 total, 8 up, 8 in 2024-02-14T17:47:09.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:09.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:09 smithi184 ceph-mon[33536]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:09.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:09.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:09 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:10.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[30321]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]': finished 2024-02-14T17:47:10.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:10.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[30321]: osdmap e214: 8 total, 8 up, 8 in 2024-02-14T17:47:10.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:10.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:10.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:10.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[30321]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:10.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3418439907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:10.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[30321]: from='client.56729 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:10.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[30321]: pgmap v279: 292 pgs: 64 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:10.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[39811]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]': finished 2024-02-14T17:47:10.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:10.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[39811]: osdmap e214: 8 total, 8 up, 8 in 2024-02-14T17:47:10.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:10.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:10.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:10.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[39811]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:10.836 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3418439907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:10.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[39811]: from='client.56729 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:10.837 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:10 smithi178 ceph-mon[39811]: pgmap v279: 292 pgs: 64 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:10.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:10 smithi184 ceph-mon[33536]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi178-80647-33"}]': finished 2024-02-14T17:47:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:10 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:10 smithi184 ceph-mon[33536]: osdmap e214: 8 total, 8 up, 8 in 2024-02-14T17:47:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/912751926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:10 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23"}]: dispatch 2024-02-14T17:47:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:10 smithi184 ceph-mon[33536]: from='client.49171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]: dispatch 2024-02-14T17:47:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3418439907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:10 smithi184 ceph-mon[33536]: from='client.56729 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:10 smithi184 ceph-mon[33536]: pgmap v279: 292 pgs: 64 unknown, 228 active+clean; 584 KiB data, 526 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:11.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:47:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23"}]': finished 2024-02-14T17:47:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]': finished 2024-02-14T17:47:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: from='client.56729 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: osdmap e215: 8 total, 8 up, 8 in 2024-02-14T17:47:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi178-80647-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:11.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi178-80647-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:11.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23"}]': finished 2024-02-14T17:47:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]': finished 2024-02-14T17:47:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: from='client.56729 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: osdmap e215: 8 total, 8 up, 8 in 2024-02-14T17:47:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi178-80647-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:11.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi178-80647-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:11.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:11 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-23"}]': finished 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: from='client.49171 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi178-80647-33"}]': finished 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: from='client.56729 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: osdmap e215: 8 total, 8 up, 8 in 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi178-80647-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi178-80647-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:11 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:12.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[30321]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi178-80647-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[30321]: osdmap e216: 8 total, 8 up, 8 in 2024-02-14T17:47:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi178-80647-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3726060443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[30321]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi178-80647-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[30321]: from='client.49195 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[30321]: pgmap v282: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 584 KiB data, 535 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:47:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[39811]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi178-80647-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[39811]: osdmap e216: 8 total, 8 up, 8 in 2024-02-14T17:47:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi178-80647-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3726060443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[39811]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi178-80647-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:12.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[39811]: from='client.49195 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:12.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:12 smithi178 ceph-mon[39811]: pgmap v282: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 584 KiB data, 535 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:47:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:12 smithi184 ceph-mon[33536]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi178-80647-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:12 smithi184 ceph-mon[33536]: osdmap e216: 8 total, 8 up, 8 in 2024-02-14T17:47:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi178-80647-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3726060443' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:12 smithi184 ceph-mon[33536]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi178-80647-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:12 smithi184 ceph-mon[33536]: from='client.49195 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:12 smithi184 ceph-mon[33536]: pgmap v282: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 584 KiB data, 535 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:47:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:13 smithi178 ceph-mon[39811]: from='client.49195 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:13 smithi178 ceph-mon[39811]: osdmap e217: 8 total, 8 up, 8 in 2024-02-14T17:47:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:13 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:13 smithi178 ceph-mon[30321]: from='client.49195 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:13 smithi178 ceph-mon[30321]: osdmap e217: 8 total, 8 up, 8 in 2024-02-14T17:47:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:13 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:13 smithi184 ceph-mon[33536]: from='client.49195 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:13 smithi184 ceph-mon[33536]: osdmap e217: 8 total, 8 up, 8 in 2024-02-14T17:47:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:13 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[39811]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi178-80647-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi178-80647-34"}]': finished 2024-02-14T17:47:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[39811]: osdmap e218: 8 total, 8 up, 8 in 2024-02-14T17:47:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1595870822' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[39811]: from='client.56747 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[39811]: pgmap v285: 364 pgs: 104 unknown, 32 creating+peering, 228 active+clean; 584 KiB data, 535 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:47:14.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[30321]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi178-80647-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi178-80647-34"}]': finished 2024-02-14T17:47:14.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:14.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[30321]: osdmap e218: 8 total, 8 up, 8 in 2024-02-14T17:47:14.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1595870822' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:14.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[30321]: from='client.56747 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:14.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:14 smithi178 ceph-mon[30321]: pgmap v285: 364 pgs: 104 unknown, 32 creating+peering, 228 active+clean; 584 KiB data, 535 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:47:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:14 smithi184 ceph-mon[33536]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi178-80647-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi178-80647-34"}]': finished 2024-02-14T17:47:14.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:14 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:14.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:14 smithi184 ceph-mon[33536]: osdmap e218: 8 total, 8 up, 8 in 2024-02-14T17:47:14.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1595870822' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:14.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:14 smithi184 ceph-mon[33536]: from='client.56747 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:14.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:14 smithi184 ceph-mon[33536]: pgmap v285: 364 pgs: 104 unknown, 32 creating+peering, 228 active+clean; 584 KiB data, 535 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:47:15.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:47:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:15.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:15.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: from='client.56747 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:15.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: osdmap e219: 8 total, 8 up, 8 in 2024-02-14T17:47:15.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:15.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:15.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: osdmap e220: 8 total, 8 up, 8 in 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/294519656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[39811]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: from='client.56747 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: osdmap e219: 8 total, 8 up, 8 in 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:15.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:15.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: osdmap e220: 8 total, 8 up, 8 in 2024-02-14T17:47:15.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:15.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/294519656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:15.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:15.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:15.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:15 smithi178 ceph-mon[30321]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: from='client.56747 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: osdmap e219: 8 total, 8 up, 8 in 2024-02-14T17:47:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: osdmap e220: 8 total, 8 up, 8 in 2024-02-14T17:47:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/294519656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:15.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:15 smithi184 ceph-mon[33536]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[30321]: pgmap v288: 388 pgs: 23 creating+peering, 9 unknown, 356 active+clean; 584 KiB data, 544 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/294519656' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-25"}]': finished 2024-02-14T17:47:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[30321]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]': finished 2024-02-14T17:47:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[30321]: osdmap e221: 8 total, 8 up, 8 in 2024-02-14T17:47:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-25", "mode": "writeback"}]: dispatch 2024-02-14T17:47:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[30321]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-25", "mode": "writeback"}]: dispatch 2024-02-14T17:47:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[39811]: pgmap v288: 388 pgs: 23 creating+peering, 9 unknown, 356 active+clean; 584 KiB data, 544 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/294519656' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-25"}]': finished 2024-02-14T17:47:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[39811]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]': finished 2024-02-14T17:47:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[39811]: osdmap e221: 8 total, 8 up, 8 in 2024-02-14T17:47:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-25", "mode": "writeback"}]: dispatch 2024-02-14T17:47:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[39811]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:16.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-25", "mode": "writeback"}]: dispatch 2024-02-14T17:47:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:16 smithi184 ceph-mon[33536]: pgmap v288: 388 pgs: 23 creating+peering, 9 unknown, 356 active+clean; 584 KiB data, 544 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/294519656' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi178-80663-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-25"}]': finished 2024-02-14T17:47:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:16 smithi184 ceph-mon[33536]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi178-80647-34"}]': finished 2024-02-14T17:47:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:16 smithi184 ceph-mon[33536]: osdmap e221: 8 total, 8 up, 8 in 2024-02-14T17:47:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/141142626' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-25", "mode": "writeback"}]: dispatch 2024-02-14T17:47:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:16 smithi184 ceph-mon[33536]: from='client.49189 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]: dispatch 2024-02-14T17:47:16.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-25", "mode": "writeback"}]: dispatch 2024-02-14T17:47:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[39811]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]': finished 2024-02-14T17:47:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-25", "mode": "writeback"}]': finished 2024-02-14T17:47:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[39811]: osdmap e222: 8 total, 8 up, 8 in 2024-02-14T17:47:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi178-80647-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[30321]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]': finished 2024-02-14T17:47:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-25", "mode": "writeback"}]': finished 2024-02-14T17:47:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[30321]: osdmap e222: 8 total, 8 up, 8 in 2024-02-14T17:47:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi178-80647-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:17 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:17 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:17 smithi184 ceph-mon[33536]: from='client.49189 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi178-80647-34"}]': finished 2024-02-14T17:47:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:17 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-25", "mode": "writeback"}]': finished 2024-02-14T17:47:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:17 smithi184 ceph-mon[33536]: osdmap e222: 8 total, 8 up, 8 in 2024-02-14T17:47:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi178-80647-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:17.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:17 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[39811]: pgmap v291: 356 pgs: 356 active+clean; 584 KiB data, 544 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi178-80647-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[39811]: osdmap e223: 8 total, 8 up, 8 in 2024-02-14T17:47:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi178-80647-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:18.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:18.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[30321]: pgmap v291: 356 pgs: 356 active+clean; 584 KiB data, 544 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi178-80647-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[30321]: osdmap e223: 8 total, 8 up, 8 in 2024-02-14T17:47:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi178-80647-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:18 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:18 smithi184 ceph-mon[33536]: pgmap v291: 356 pgs: 356 active+clean; 584 KiB data, 544 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:47:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi178-80647-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:18 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:18 smithi184 ceph-mon[33536]: osdmap e223: 8 total, 8 up, 8 in 2024-02-14T17:47:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi178-80647-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:18 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25"}]: dispatch 2024-02-14T17:47:19.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:19 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:19.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25"}]': finished 2024-02-14T17:47:20.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:19 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:20.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25"}]': finished 2024-02-14T17:47:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:19 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-25"}]': finished 2024-02-14T17:47:20.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:20 smithi184 ceph-mon[33536]: osdmap e224: 8 total, 8 up, 8 in 2024-02-14T17:47:20.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:20 smithi184 ceph-mon[33536]: pgmap v294: 292 pgs: 292 active+clean; 584 KiB data, 544 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:20.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:20 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:20.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi178-80647-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi178-80647-35"}]': finished 2024-02-14T17:47:20.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:20 smithi184 ceph-mon[33536]: osdmap e225: 8 total, 8 up, 8 in 2024-02-14T17:47:21.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:20 smithi178 ceph-mon[39811]: osdmap e224: 8 total, 8 up, 8 in 2024-02-14T17:47:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:20 smithi178 ceph-mon[39811]: pgmap v294: 292 pgs: 292 active+clean; 584 KiB data, 544 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:20 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi178-80647-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi178-80647-35"}]': finished 2024-02-14T17:47:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:20 smithi178 ceph-mon[39811]: osdmap e225: 8 total, 8 up, 8 in 2024-02-14T17:47:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:20 smithi178 ceph-mon[30321]: osdmap e224: 8 total, 8 up, 8 in 2024-02-14T17:47:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:20 smithi178 ceph-mon[30321]: pgmap v294: 292 pgs: 292 active+clean; 584 KiB data, 544 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:20 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi178-80647-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi178-80647-35"}]': finished 2024-02-14T17:47:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:20 smithi178 ceph-mon[30321]: osdmap e225: 8 total, 8 up, 8 in 2024-02-14T17:47:21.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:47:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:47:21.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:21 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:21.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:21 smithi184 ceph-mon[33536]: osdmap e226: 8 total, 8 up, 8 in 2024-02-14T17:47:21.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:21.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:21 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:22.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:21 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:21 smithi178 ceph-mon[30321]: osdmap e226: 8 total, 8 up, 8 in 2024-02-14T17:47:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:21 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:21 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:21 smithi178 ceph-mon[39811]: osdmap e226: 8 total, 8 up, 8 in 2024-02-14T17:47:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:21 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:22.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:22 smithi184 ceph-mon[33536]: pgmap v297: 236 pgs: 7 creating+peering, 33 unknown, 196 active+clean; 583 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:22.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:22 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:22.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:22 smithi184 ceph-mon[33536]: osdmap e227: 8 total, 8 up, 8 in 2024-02-14T17:47:22.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2670274494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi178-80663-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:22.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:23.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:22 smithi178 ceph-mon[39811]: pgmap v297: 236 pgs: 7 creating+peering, 33 unknown, 196 active+clean; 583 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:22 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:22 smithi178 ceph-mon[39811]: osdmap e227: 8 total, 8 up, 8 in 2024-02-14T17:47:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2670274494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi178-80663-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:22 smithi178 ceph-mon[30321]: pgmap v297: 236 pgs: 7 creating+peering, 33 unknown, 196 active+clean; 583 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:22 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:22 smithi178 ceph-mon[30321]: osdmap e227: 8 total, 8 up, 8 in 2024-02-14T17:47:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2670274494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi178-80663-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:24.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2670274494' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi178-80663-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:24.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi178-80647-35"}]': finished 2024-02-14T17:47:24.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:24 smithi184 ceph-mon[33536]: osdmap e228: 8 total, 8 up, 8 in 2024-02-14T17:47:24.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:24.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:24.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:24 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:24.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:24 smithi184 ceph-mon[33536]: pgmap v300: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:24.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2670274494' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi178-80663-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi178-80647-35"}]': finished 2024-02-14T17:47:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[39811]: osdmap e228: 8 total, 8 up, 8 in 2024-02-14T17:47:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[39811]: pgmap v300: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2670274494' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi178-80663-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi178-80647-35"}]': finished 2024-02-14T17:47:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[30321]: osdmap e228: 8 total, 8 up, 8 in 2024-02-14T17:47:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi178-80647-35"}]: dispatch 2024-02-14T17:47:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:24.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:24.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:24 smithi178 ceph-mon[30321]: pgmap v300: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:25.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi178-80647-35"}]': finished 2024-02-14T17:47:25.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:25.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[39811]: osdmap e229: 8 total, 8 up, 8 in 2024-02-14T17:47:25.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:25.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:25.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[39811]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[39811]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi178-80647-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:25.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[39811]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi178-80647-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:25.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi178-80647-35"}]': finished 2024-02-14T17:47:25.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:25.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[30321]: osdmap e229: 8 total, 8 up, 8 in 2024-02-14T17:47:25.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:25.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:25.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[30321]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[30321]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi178-80647-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:25.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-mon[30321]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi178-80647-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:25.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/700532324' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi178-80647-35"}]': finished 2024-02-14T17:47:25.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:25 smithi184 ceph-mon[33536]: osdmap e229: 8 total, 8 up, 8 in 2024-02-14T17:47:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:25 smithi184 ceph-mon[33536]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:25 smithi184 ceph-mon[33536]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi178-80647-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:25 smithi184 ceph-mon[33536]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi178-80647-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:26.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:47:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:47:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-27"}]': finished 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi178-80647-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: osdmap e230: 8 total, 8 up, 8 in 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi178-80647-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-27", "mode": "writeback"}]: dispatch 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2565180776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi178-80663-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi178-80647-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: from='client.49225 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi178-80663-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-27", "mode": "writeback"}]: dispatch 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: pgmap v303: 260 pgs: 25 creating+peering, 7 unknown, 228 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:26.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:26 smithi184 ceph-mon[33536]: from='client.49225 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi178-80663-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:27.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-27"}]': finished 2024-02-14T17:47:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi178-80647-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: osdmap e230: 8 total, 8 up, 8 in 2024-02-14T17:47:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi178-80647-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-27", "mode": "writeback"}]: dispatch 2024-02-14T17:47:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2565180776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi178-80663-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi178-80647-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: from='client.49225 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi178-80663-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-27", "mode": "writeback"}]: dispatch 2024-02-14T17:47:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: pgmap v303: 260 pgs: 25 creating+peering, 7 unknown, 228 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:47:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[30321]: from='client.49225 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi178-80663-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-27"}]': finished 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi178-80647-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: osdmap e230: 8 total, 8 up, 8 in 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi178-80647-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-27", "mode": "writeback"}]: dispatch 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2565180776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi178-80663-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi178-80647-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: from='client.49225 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi178-80663-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-27", "mode": "writeback"}]: dispatch 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: pgmap v303: 260 pgs: 25 creating+peering, 7 unknown, 228 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:27.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:26 smithi178 ceph-mon[39811]: from='client.49225 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi178-80663-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:27 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-27", "mode": "writeback"}]': finished 2024-02-14T17:47:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:27 smithi184 ceph-mon[33536]: osdmap e231: 8 total, 8 up, 8 in 2024-02-14T17:47:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:27 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:27 smithi184 ceph-mon[33536]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi178-80647-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi178-80647-36"}]': finished 2024-02-14T17:47:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:27 smithi184 ceph-mon[33536]: osdmap e232: 8 total, 8 up, 8 in 2024-02-14T17:47:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:27.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:27 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-27", "mode": "writeback"}]': finished 2024-02-14T17:47:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[39811]: osdmap e231: 8 total, 8 up, 8 in 2024-02-14T17:47:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[39811]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi178-80647-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi178-80647-36"}]': finished 2024-02-14T17:47:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[39811]: osdmap e232: 8 total, 8 up, 8 in 2024-02-14T17:47:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:28.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:28.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:28.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-27", "mode": "writeback"}]': finished 2024-02-14T17:47:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[30321]: osdmap e231: 8 total, 8 up, 8 in 2024-02-14T17:47:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[30321]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi178-80647-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi178-80647-36"}]': finished 2024-02-14T17:47:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[30321]: osdmap e232: 8 total, 8 up, 8 in 2024-02-14T17:47:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:28.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:28.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:27 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:29.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[39811]: pgmap v306: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:47:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[39811]: osdmap e233: 8 total, 8 up, 8 in 2024-02-14T17:47:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[30321]: pgmap v306: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:47:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[30321]: osdmap e233: 8 total, 8 up, 8 in 2024-02-14T17:47:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:29.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:28 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:29.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:28 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:28 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:28 smithi184 ceph-mon[33536]: pgmap v306: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:47:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:28 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:28 smithi184 ceph-mon[33536]: osdmap e233: 8 total, 8 up, 8 in 2024-02-14T17:47:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:29.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:28 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27"}]: dispatch 2024-02-14T17:47:30.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:29 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:29 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27"}]': finished 2024-02-14T17:47:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:29 smithi178 ceph-mon[39811]: osdmap e234: 8 total, 8 up, 8 in 2024-02-14T17:47:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:29 smithi178 ceph-mon[39811]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:29 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:29 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27"}]': finished 2024-02-14T17:47:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:29 smithi178 ceph-mon[30321]: osdmap e234: 8 total, 8 up, 8 in 2024-02-14T17:47:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:29 smithi178 ceph-mon[30321]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:30.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:29 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:30.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:29 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-27"}]': finished 2024-02-14T17:47:30.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:29 smithi184 ceph-mon[33536]: osdmap e234: 8 total, 8 up, 8 in 2024-02-14T17:47:30.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:30.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:29 smithi184 ceph-mon[33536]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:31.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[39811]: pgmap v309: 228 pgs: 228 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:47:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-46"}]': finished 2024-02-14T17:47:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[39811]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]': finished 2024-02-14T17:47:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[39811]: osdmap e235: 8 total, 8 up, 8 in 2024-02-14T17:47:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[39811]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[30321]: pgmap v309: 228 pgs: 228 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:47:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-46"}]': finished 2024-02-14T17:47:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[30321]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]': finished 2024-02-14T17:47:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[30321]: osdmap e235: 8 total, 8 up, 8 in 2024-02-14T17:47:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:31.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:30 smithi178 ceph-mon[30321]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:31.090 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:30 smithi184 ceph-mon[33536]: pgmap v309: 228 pgs: 228 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:31.090 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:30 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:47:31.090 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-46"}]': finished 2024-02-14T17:47:31.090 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:30 smithi184 ceph-mon[33536]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi178-80647-36"}]': finished 2024-02-14T17:47:31.090 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:30 smithi184 ceph-mon[33536]: osdmap e235: 8 total, 8 up, 8 in 2024-02-14T17:47:31.090 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3074415002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:31.090 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:30 smithi184 ceph-mon[33536]: from='client.56777 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]: dispatch 2024-02-14T17:47:31.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:47:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:47:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:47:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]': finished 2024-02-14T17:47:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: osdmap e236: 8 total, 8 up, 8 in 2024-02-14T17:47:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:32.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:32.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[39811]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:32.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:47:32.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:47:32.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:32.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]': finished 2024-02-14T17:47:32.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: osdmap e236: 8 total, 8 up, 8 in 2024-02-14T17:47:32.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:32.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:32.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:32.086 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:31 smithi178 ceph-mon[30321]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:47:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:47:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='client.56777 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi178-80647-36"}]': finished 2024-02-14T17:47:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: osdmap e236: 8 total, 8 up, 8 in 2024-02-14T17:47:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:32.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:32.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:32.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:31 smithi184 ceph-mon[33536]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[39811]: pgmap v312: 236 pgs: 7 creating+peering, 33 unknown, 196 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[39811]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[39811]: osdmap e237: 8 total, 8 up, 8 in 2024-02-14T17:47:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi178-80647-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[39811]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi178-80647-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:33.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[30321]: pgmap v312: 236 pgs: 7 creating+peering, 33 unknown, 196 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:33.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:33.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[30321]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:33.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[30321]: osdmap e237: 8 total, 8 up, 8 in 2024-02-14T17:47:33.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:33.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi178-80647-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:33.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:32 smithi178 ceph-mon[30321]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi178-80647-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:33.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:32 smithi184 ceph-mon[33536]: pgmap v312: 236 pgs: 7 creating+peering, 33 unknown, 196 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:33.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:32 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:32 smithi184 ceph-mon[33536]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:32 smithi184 ceph-mon[33536]: osdmap e237: 8 total, 8 up, 8 in 2024-02-14T17:47:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi178-80647-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:33.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:32 smithi184 ceph-mon[33536]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi178-80647-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:34.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:33 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-46"}]': finished 2024-02-14T17:47:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:33 smithi178 ceph-mon[39811]: osdmap e238: 8 total, 8 up, 8 in 2024-02-14T17:47:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:33 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-46"}]': finished 2024-02-14T17:47:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:33 smithi178 ceph-mon[30321]: osdmap e238: 8 total, 8 up, 8 in 2024-02-14T17:47:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:34.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:33 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-46"}]': finished 2024-02-14T17:47:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:33 smithi184 ceph-mon[33536]: osdmap e238: 8 total, 8 up, 8 in 2024-02-14T17:47:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-46"}]: dispatch 2024-02-14T17:47:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: pgmap v315: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi178-80647-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi178-80647-37"}]': finished 2024-02-14T17:47:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-46"}]': finished 2024-02-14T17:47:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: osdmap e239: 8 total, 8 up, 8 in 2024-02-14T17:47:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:35.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[39811]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: pgmap v315: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi178-80647-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi178-80647-37"}]': finished 2024-02-14T17:47:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-46"}]': finished 2024-02-14T17:47:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: osdmap e239: 8 total, 8 up, 8 in 2024-02-14T17:47:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:35.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:35.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:34 smithi178 ceph-mon[30321]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: pgmap v315: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi178-80647-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi178-80647-37"}]': finished 2024-02-14T17:47:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/661054590' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-46"}]': finished 2024-02-14T17:47:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: osdmap e239: 8 total, 8 up, 8 in 2024-02-14T17:47:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:35.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:35.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:34 smithi184 ceph-mon[33536]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:47:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:47:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:36 smithi184 ceph-mon[33536]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:36 smithi184 ceph-mon[33536]: osdmap e240: 8 total, 8 up, 8 in 2024-02-14T17:47:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:36 smithi184 ceph-mon[33536]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:36 smithi184 ceph-mon[33536]: pgmap v318: 236 pgs: 3 creating+peering, 233 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-14T17:47:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[30321]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[30321]: osdmap e240: 8 total, 8 up, 8 in 2024-02-14T17:47:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[30321]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[30321]: pgmap v318: 236 pgs: 3 creating+peering, 233 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-14T17:47:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[39811]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi178-80663-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[39811]: osdmap e240: 8 total, 8 up, 8 in 2024-02-14T17:47:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[39811]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:36 smithi178 ceph-mon[39811]: pgmap v318: 236 pgs: 3 creating+peering, 233 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-14T17:47:37.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:37 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-29"}]': finished 2024-02-14T17:47:37.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-29", "mode": "writeback"}]: dispatch 2024-02-14T17:47:37.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:37 smithi184 ceph-mon[33536]: osdmap e241: 8 total, 8 up, 8 in 2024-02-14T17:47:37.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:37 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-29", "mode": "writeback"}]: dispatch 2024-02-14T17:47:37.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:37.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:37 smithi184 ceph-mon[33536]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:37.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-29"}]': finished 2024-02-14T17:47:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-29", "mode": "writeback"}]: dispatch 2024-02-14T17:47:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[39811]: osdmap e241: 8 total, 8 up, 8 in 2024-02-14T17:47:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-29", "mode": "writeback"}]: dispatch 2024-02-14T17:47:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[39811]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-29"}]': finished 2024-02-14T17:47:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-29", "mode": "writeback"}]: dispatch 2024-02-14T17:47:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[30321]: osdmap e241: 8 total, 8 up, 8 in 2024-02-14T17:47:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-29", "mode": "writeback"}]: dispatch 2024-02-14T17:47:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:37.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:37 smithi178 ceph-mon[30321]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:38.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:38 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:38.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:38 smithi184 ceph-mon[33536]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-47"}]': finished 2024-02-14T17:47:38.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:38 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-29", "mode": "writeback"}]': finished 2024-02-14T17:47:38.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:38 smithi184 ceph-mon[33536]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]': finished 2024-02-14T17:47:38.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:38 smithi184 ceph-mon[33536]: osdmap e242: 8 total, 8 up, 8 in 2024-02-14T17:47:38.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:38.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:38 smithi184 ceph-mon[33536]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:38.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:38 smithi184 ceph-mon[33536]: pgmap v321: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-14T17:47:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[39811]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-47"}]': finished 2024-02-14T17:47:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-29", "mode": "writeback"}]': finished 2024-02-14T17:47:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[39811]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]': finished 2024-02-14T17:47:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[39811]: osdmap e242: 8 total, 8 up, 8 in 2024-02-14T17:47:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[39811]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[39811]: pgmap v321: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-14T17:47:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:38.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[30321]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi178-80663-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi178-80663-47"}]': finished 2024-02-14T17:47:38.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-29", "mode": "writeback"}]': finished 2024-02-14T17:47:38.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[30321]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi178-80647-37"}]': finished 2024-02-14T17:47:38.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[30321]: osdmap e242: 8 total, 8 up, 8 in 2024-02-14T17:47:38.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1855948167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:38.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[30321]: from='client.49240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]: dispatch 2024-02-14T17:47:38.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:38 smithi178 ceph-mon[30321]: pgmap v321: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-14T17:47:39.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:39 smithi184 ceph-mon[33536]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]': finished 2024-02-14T17:47:39.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:39 smithi184 ceph-mon[33536]: osdmap e243: 8 total, 8 up, 8 in 2024-02-14T17:47:39.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:39.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:39.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi178-80647-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:39.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:39 smithi178 ceph-mon[39811]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]': finished 2024-02-14T17:47:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:39 smithi178 ceph-mon[39811]: osdmap e243: 8 total, 8 up, 8 in 2024-02-14T17:47:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi178-80647-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:39 smithi178 ceph-mon[30321]: from='client.49240 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi178-80647-37"}]': finished 2024-02-14T17:47:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:39 smithi178 ceph-mon[30321]: osdmap e243: 8 total, 8 up, 8 in 2024-02-14T17:47:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi178-80647-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:40.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi178-80647-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:40 smithi184 ceph-mon[33536]: osdmap e244: 8 total, 8 up, 8 in 2024-02-14T17:47:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi178-80647-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:40 smithi184 ceph-mon[33536]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:40 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.8", "id": [6, 7]}]: dispatch 2024-02-14T17:47:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:40 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.8", "id": [6, 7]}]: dispatch 2024-02-14T17:47:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:40 smithi184 ceph-mon[33536]: pgmap v324: 228 pgs: 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:40 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi178-80647-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[30321]: osdmap e244: 8 total, 8 up, 8 in 2024-02-14T17:47:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi178-80647-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[30321]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.8", "id": [6, 7]}]: dispatch 2024-02-14T17:47:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.8", "id": [6, 7]}]: dispatch 2024-02-14T17:47:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[30321]: pgmap v324: 228 pgs: 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi178-80647-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[39811]: osdmap e244: 8 total, 8 up, 8 in 2024-02-14T17:47:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi178-80647-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[39811]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.8", "id": [6, 7]}]: dispatch 2024-02-14T17:47:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.8", "id": [6, 7]}]: dispatch 2024-02-14T17:47:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[39811]: pgmap v324: 228 pgs: 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:40 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:41.417 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:47:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:47:41.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:41 smithi184 ceph-mon[33536]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]': finished 2024-02-14T17:47:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:41 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.8", "id": [6, 7]}]': finished 2024-02-14T17:47:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:41 smithi184 ceph-mon[33536]: osdmap e245: 8 total, 8 up, 8 in 2024-02-14T17:47:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:41 smithi184 ceph-mon[33536]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:41 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:41 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[39811]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]': finished 2024-02-14T17:47:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.8", "id": [6, 7]}]': finished 2024-02-14T17:47:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[39811]: osdmap e245: 8 total, 8 up, 8 in 2024-02-14T17:47:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[39811]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[30321]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi178-80663-47"}]': finished 2024-02-14T17:47:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.8", "id": [6, 7]}]': finished 2024-02-14T17:47:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/374127502' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[30321]: osdmap e245: 8 total, 8 up, 8 in 2024-02-14T17:47:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[30321]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]: dispatch 2024-02-14T17:47:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:41 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi178-80647-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi178-80647-38"}]': finished 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]': finished 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: osdmap e246: 8 total, 8 up, 8 in 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi178-80663-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi178-80663-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:42 smithi184 ceph-mon[33536]: pgmap v327: 236 pgs: 8 unknown, 1 peering, 227 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi178-80647-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi178-80647-38"}]': finished 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]': finished 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: osdmap e246: 8 total, 8 up, 8 in 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi178-80663-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi178-80663-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[39811]: pgmap v327: 236 pgs: 8 unknown, 1 peering, 227 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi178-80647-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi178-80647-38"}]': finished 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi178-80663-47"}]': finished 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: osdmap e246: 8 total, 8 up, 8 in 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29"}]: dispatch 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi178-80663-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:42.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi178-80663-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:42.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:42 smithi178 ceph-mon[30321]: pgmap v327: 236 pgs: 8 unknown, 1 peering, 227 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-02-14T17:47:43.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:43 smithi184 ceph-mon[33536]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-14T17:47:43.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:43 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:43.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29"}]': finished 2024-02-14T17:47:43.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:43 smithi184 ceph-mon[33536]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi178-80663-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:43.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:43 smithi184 ceph-mon[33536]: osdmap e247: 8 total, 8 up, 8 in 2024-02-14T17:47:43.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi178-80663-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:43.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:43 smithi184 ceph-mon[33536]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi178-80663-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:43.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:43 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:43.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[39811]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-14T17:47:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29"}]': finished 2024-02-14T17:47:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[39811]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi178-80663-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[39811]: osdmap e247: 8 total, 8 up, 8 in 2024-02-14T17:47:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi178-80663-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[39811]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi178-80663-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:44.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[30321]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-14T17:47:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-29"}]': finished 2024-02-14T17:47:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[30321]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi178-80663-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[30321]: osdmap e247: 8 total, 8 up, 8 in 2024-02-14T17:47:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi178-80663-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[30321]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi178-80663-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:44.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:44 smithi184 ceph-mon[33536]: osdmap e248: 8 total, 8 up, 8 in 2024-02-14T17:47:44.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:44.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:44 smithi184 ceph-mon[33536]: pgmap v330: 196 pgs: 1 peering, 195 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:44.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:44 smithi184 ceph-mon[33536]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi178-80663-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi178-80663-48"}]': finished 2024-02-14T17:47:44.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi178-80647-38"}]': finished 2024-02-14T17:47:44.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:44 smithi184 ceph-mon[33536]: osdmap e249: 8 total, 8 up, 8 in 2024-02-14T17:47:44.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:44.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:44.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:44 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:45.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[39811]: osdmap e248: 8 total, 8 up, 8 in 2024-02-14T17:47:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[39811]: pgmap v330: 196 pgs: 1 peering, 195 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[39811]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi178-80663-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi178-80663-48"}]': finished 2024-02-14T17:47:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi178-80647-38"}]': finished 2024-02-14T17:47:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[39811]: osdmap e249: 8 total, 8 up, 8 in 2024-02-14T17:47:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[30321]: osdmap e248: 8 total, 8 up, 8 in 2024-02-14T17:47:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[30321]: pgmap v330: 196 pgs: 1 peering, 195 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[30321]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi178-80663-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi178-80663-48"}]': finished 2024-02-14T17:47:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi178-80647-38"}]': finished 2024-02-14T17:47:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[30321]: osdmap e249: 8 total, 8 up, 8 in 2024-02-14T17:47:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi178-80647-38"}]: dispatch 2024-02-14T17:47:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:44 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:45 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi178-80647-38"}]': finished 2024-02-14T17:47:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:45 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:45 smithi184 ceph-mon[33536]: osdmap e250: 8 total, 8 up, 8 in 2024-02-14T17:47:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:45 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:45.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:45.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi178-80647-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:45.984 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:47:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi178-80647-38"}]': finished 2024-02-14T17:47:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[30321]: osdmap e250: 8 total, 8 up, 8 in 2024-02-14T17:47:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi178-80647-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3986917614' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi178-80647-38"}]': finished 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[39811]: osdmap e250: 8 total, 8 up, 8 in 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi178-80647-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:46.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:46 smithi184 ceph-mon[33536]: pgmap v333: 236 pgs: 10 creating+peering, 226 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 5 B/s, 2 objects/s recovering 2024-02-14T17:47:46.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:46 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi178-80647-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:46 smithi184 ceph-mon[33536]: osdmap e251: 8 total, 8 up, 8 in 2024-02-14T17:47:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi178-80647-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:46 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:46 smithi184 ceph-mon[33536]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[39811]: pgmap v333: 236 pgs: 10 creating+peering, 226 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 5 B/s, 2 objects/s recovering 2024-02-14T17:47:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi178-80647-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[39811]: osdmap e251: 8 total, 8 up, 8 in 2024-02-14T17:47:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi178-80647-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[39811]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[30321]: pgmap v333: 236 pgs: 10 creating+peering, 226 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 5 B/s, 2 objects/s recovering 2024-02-14T17:47:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi178-80647-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[30321]: osdmap e251: 8 total, 8 up, 8 in 2024-02-14T17:47:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi178-80647-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:46 smithi178 ceph-mon[30321]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:47 smithi184 ceph-mon[33536]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-14T17:47:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:47 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-31"}]': finished 2024-02-14T17:47:47.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:47 smithi184 ceph-mon[33536]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]': finished 2024-02-14T17:47:47.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-31", "mode": "writeback"}]: dispatch 2024-02-14T17:47:47.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:47.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:47 smithi184 ceph-mon[33536]: osdmap e252: 8 total, 8 up, 8 in 2024-02-14T17:47:47.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:47 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-31", "mode": "writeback"}]: dispatch 2024-02-14T17:47:47.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:47 smithi184 ceph-mon[33536]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:48.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[39811]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-31"}]': finished 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[39811]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]': finished 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-31", "mode": "writeback"}]: dispatch 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[39811]: osdmap e252: 8 total, 8 up, 8 in 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-31", "mode": "writeback"}]: dispatch 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[39811]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[30321]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-31"}]': finished 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[30321]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi178-80663-48"}]': finished 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-31", "mode": "writeback"}]: dispatch 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2581107438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[30321]: osdmap e252: 8 total, 8 up, 8 in 2024-02-14T17:47:48.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-31", "mode": "writeback"}]: dispatch 2024-02-14T17:47:48.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:47 smithi178 ceph-mon[30321]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]: dispatch 2024-02-14T17:47:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: pgmap v336: 228 pgs: 9 creating+peering, 219 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s; 5 B/s, 2 objects/s recovering 2024-02-14T17:47:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi178-80647-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi178-80647-39"}]': finished 2024-02-14T17:47:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-31", "mode": "writeback"}]': finished 2024-02-14T17:47:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]': finished 2024-02-14T17:47:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: osdmap e253: 8 total, 8 up, 8 in 2024-02-14T17:47:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi178-80663-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi178-80663-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:48.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: pgmap v336: 228 pgs: 9 creating+peering, 219 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s; 5 B/s, 2 objects/s recovering 2024-02-14T17:47:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi178-80647-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi178-80647-39"}]': finished 2024-02-14T17:47:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-31", "mode": "writeback"}]': finished 2024-02-14T17:47:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]': finished 2024-02-14T17:47:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: osdmap e253: 8 total, 8 up, 8 in 2024-02-14T17:47:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi178-80663-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi178-80663-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:49.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: pgmap v336: 228 pgs: 9 creating+peering, 219 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s; 5 B/s, 2 objects/s recovering 2024-02-14T17:47:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi178-80647-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi178-80647-39"}]': finished 2024-02-14T17:47:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-31", "mode": "writeback"}]': finished 2024-02-14T17:47:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi178-80663-48"}]': finished 2024-02-14T17:47:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: osdmap e253: 8 total, 8 up, 8 in 2024-02-14T17:47:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi178-80663-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi178-80663-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:49.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:50 smithi184 ceph-mon[33536]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi178-80663-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:50.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:50 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:50.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:50 smithi184 ceph-mon[33536]: osdmap e254: 8 total, 8 up, 8 in 2024-02-14T17:47:50.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:50.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi178-80663-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:50.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:50 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:50.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:50 smithi184 ceph-mon[33536]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi178-80663-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:50.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:50 smithi184 ceph-mon[33536]: pgmap v339: 236 pgs: 8 unknown, 9 creating+peering, 219 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:50.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:50 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:50.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:50 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31"}]': finished 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[39811]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi178-80663-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[39811]: osdmap e254: 8 total, 8 up, 8 in 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi178-80663-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[39811]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi178-80663-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[39811]: pgmap v339: 236 pgs: 8 unknown, 9 creating+peering, 219 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31"}]': finished 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[30321]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi178-80663-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[30321]: osdmap e254: 8 total, 8 up, 8 in 2024-02-14T17:47:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi178-80663-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31"}]: dispatch 2024-02-14T17:47:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[30321]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi178-80663-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[30321]: pgmap v339: 236 pgs: 8 unknown, 9 creating+peering, 219 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:50 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-31"}]': finished 2024-02-14T17:47:51.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:47:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:47:51.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:51 smithi184 ceph-mon[33536]: osdmap e255: 8 total, 8 up, 8 in 2024-02-14T17:47:51.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:51 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:51.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:51.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:51 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:51.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:51 smithi184 ceph-mon[33536]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi178-80663-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi178-80663-49"}]': finished 2024-02-14T17:47:51.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi178-80647-39"}]': finished 2024-02-14T17:47:51.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:51 smithi184 ceph-mon[33536]: osdmap e256: 8 total, 8 up, 8 in 2024-02-14T17:47:51.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[39811]: osdmap e255: 8 total, 8 up, 8 in 2024-02-14T17:47:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[39811]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi178-80663-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi178-80663-49"}]': finished 2024-02-14T17:47:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi178-80647-39"}]': finished 2024-02-14T17:47:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[39811]: osdmap e256: 8 total, 8 up, 8 in 2024-02-14T17:47:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[30321]: osdmap e255: 8 total, 8 up, 8 in 2024-02-14T17:47:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:52.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:52.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[30321]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi178-80663-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi178-80663-49"}]': finished 2024-02-14T17:47:52.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi178-80647-39"}]': finished 2024-02-14T17:47:52.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[30321]: osdmap e256: 8 total, 8 up, 8 in 2024-02-14T17:47:52.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi178-80647-39"}]: dispatch 2024-02-14T17:47:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:52 smithi184 ceph-mon[33536]: pgmap v342: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:47:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi178-80647-39"}]': finished 2024-02-14T17:47:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:52 smithi184 ceph-mon[33536]: osdmap e257: 8 total, 8 up, 8 in 2024-02-14T17:47:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:52 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:52 smithi184 ceph-mon[33536]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:52 smithi184 ceph-mon[33536]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi178-80647-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:52 smithi184 ceph-mon[33536]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi178-80647-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[39811]: pgmap v342: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:47:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi178-80647-39"}]': finished 2024-02-14T17:47:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[39811]: osdmap e257: 8 total, 8 up, 8 in 2024-02-14T17:47:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[39811]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:53.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[39811]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:53.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi178-80647-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:53.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[39811]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi178-80647-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:53.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[30321]: pgmap v342: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:47:53.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/195552361' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi178-80647-39"}]': finished 2024-02-14T17:47:53.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[30321]: osdmap e257: 8 total, 8 up, 8 in 2024-02-14T17:47:53.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:53.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[30321]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:53.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:53.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[30321]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:53.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi178-80647-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:53.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:52 smithi178 ceph-mon[30321]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi178-80647-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:54 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:54 smithi184 ceph-mon[33536]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi178-80647-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:54 smithi184 ceph-mon[33536]: osdmap e258: 8 total, 8 up, 8 in 2024-02-14T17:47:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi178-80647-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:54 smithi184 ceph-mon[33536]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi178-80647-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:54 smithi184 ceph-mon[33536]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:54 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:54.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:54 smithi184 ceph-mon[33536]: pgmap v345: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:47:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[39811]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi178-80647-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[39811]: osdmap e258: 8 total, 8 up, 8 in 2024-02-14T17:47:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi178-80647-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[39811]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi178-80647-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[39811]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[39811]: pgmap v345: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:47:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[30321]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi178-80647-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[30321]: osdmap e258: 8 total, 8 up, 8 in 2024-02-14T17:47:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi178-80647-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[30321]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi178-80647-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[30321]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:47:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:54 smithi178 ceph-mon[30321]: pgmap v345: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:47:55.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[39811]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]': finished 2024-02-14T17:47:55.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:55.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:55.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:55.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[39811]: osdmap e259: 8 total, 8 up, 8 in 2024-02-14T17:47:55.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:55.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[39811]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:55.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[30321]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]': finished 2024-02-14T17:47:55.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:55.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:55.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:55.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[30321]: osdmap e259: 8 total, 8 up, 8 in 2024-02-14T17:47:55.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:55.691 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-mon[30321]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:55.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:55 smithi184 ceph-mon[33536]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi178-80663-49"}]': finished 2024-02-14T17:47:55.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:55 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:47:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/558925668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:55 smithi184 ceph-mon[33536]: osdmap e259: 8 total, 8 up, 8 in 2024-02-14T17:47:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:55 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:55.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:55 smithi184 ceph-mon[33536]: from='client.56819 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]: dispatch 2024-02-14T17:47:56.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:47:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:47:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi178-80647-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi178-80647-40"}]': finished 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-33"}]': finished 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]': finished 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: osdmap e260: 8 total, 8 up, 8 in 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-33", "mode": "writeback"}]: dispatch 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-33", "mode": "writeback"}]: dispatch 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi178-80663-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi178-80663-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:56 smithi184 ceph-mon[33536]: pgmap v348: 236 pgs: 5 creating+peering, 3 unknown, 228 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi178-80647-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi178-80647-40"}]': finished 2024-02-14T17:47:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-33"}]': finished 2024-02-14T17:47:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]': finished 2024-02-14T17:47:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: osdmap e260: 8 total, 8 up, 8 in 2024-02-14T17:47:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-33", "mode": "writeback"}]: dispatch 2024-02-14T17:47:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-33", "mode": "writeback"}]: dispatch 2024-02-14T17:47:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi178-80663-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi178-80663-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[39811]: pgmap v348: 236 pgs: 5 creating+peering, 3 unknown, 228 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi178-80647-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi178-80647-40"}]': finished 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-33"}]': finished 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: from='client.56819 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi178-80663-49"}]': finished 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: osdmap e260: 8 total, 8 up, 8 in 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-33", "mode": "writeback"}]: dispatch 2024-02-14T17:47:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-33", "mode": "writeback"}]: dispatch 2024-02-14T17:47:56.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:56.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi178-80663-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:56.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi178-80663-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:56.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:56 smithi178 ceph-mon[30321]: pgmap v348: 236 pgs: 5 creating+peering, 3 unknown, 228 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:57 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:57 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:57 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-33", "mode": "writeback"}]': finished 2024-02-14T17:47:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:57 smithi184 ceph-mon[33536]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi178-80663-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:57 smithi184 ceph-mon[33536]: osdmap e261: 8 total, 8 up, 8 in 2024-02-14T17:47:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi178-80663-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:57 smithi184 ceph-mon[33536]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi178-80663-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:57 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-33", "mode": "writeback"}]': finished 2024-02-14T17:47:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[30321]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi178-80663-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[30321]: osdmap e261: 8 total, 8 up, 8 in 2024-02-14T17:47:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi178-80663-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[30321]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi178-80663-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:57.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:57.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:47:57.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-33", "mode": "writeback"}]': finished 2024-02-14T17:47:57.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[39811]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi178-80663-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:57.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[39811]: osdmap e261: 8 total, 8 up, 8 in 2024-02-14T17:47:57.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi178-80663-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:57.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[39811]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi178-80663-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:47:57.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:57.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:57 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:47:58.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:58 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:58 smithi184 ceph-mon[33536]: osdmap e262: 8 total, 8 up, 8 in 2024-02-14T17:47:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:58 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:58 smithi184 ceph-mon[33536]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:58 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:58 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:58 smithi184 ceph-mon[33536]: pgmap v351: 228 pgs: 228 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[30321]: osdmap e262: 8 total, 8 up, 8 in 2024-02-14T17:47:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[30321]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[30321]: pgmap v351: 228 pgs: 228 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:47:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[39811]: osdmap e262: 8 total, 8 up, 8 in 2024-02-14T17:47:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33"}]: dispatch 2024-02-14T17:47:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[39811]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:47:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:58 smithi178 ceph-mon[39811]: pgmap v351: 228 pgs: 228 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:59.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:59 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:59 smithi184 ceph-mon[33536]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi178-80663-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi178-80663-50"}]': finished 2024-02-14T17:47:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:59 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33"}]': finished 2024-02-14T17:47:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:59 smithi184 ceph-mon[33536]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]': finished 2024-02-14T17:47:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:59 smithi184 ceph-mon[33536]: osdmap e263: 8 total, 8 up, 8 in 2024-02-14T17:47:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:59.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:47:59 smithi184 ceph-mon[33536]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:59.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[39811]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi178-80663-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi178-80663-50"}]': finished 2024-02-14T17:47:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33"}]': finished 2024-02-14T17:47:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[39811]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]': finished 2024-02-14T17:47:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[39811]: osdmap e263: 8 total, 8 up, 8 in 2024-02-14T17:47:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[39811]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:47:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[30321]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi178-80663-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi178-80663-50"}]': finished 2024-02-14T17:47:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-33"}]': finished 2024-02-14T17:47:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[30321]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi178-80647-40"}]': finished 2024-02-14T17:47:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[30321]: osdmap e263: 8 total, 8 up, 8 in 2024-02-14T17:47:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3360187408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:47:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:47:59 smithi178 ceph-mon[30321]: from='client.56825 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]: dispatch 2024-02-14T17:48:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:00 smithi184 ceph-mon[33536]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]': finished 2024-02-14T17:48:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:00 smithi184 ceph-mon[33536]: osdmap e264: 8 total, 8 up, 8 in 2024-02-14T17:48:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:00 smithi184 ceph-mon[33536]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:00 smithi184 ceph-mon[33536]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi178-80647-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:00 smithi184 ceph-mon[33536]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi178-80647-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:00.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:00 smithi184 ceph-mon[33536]: pgmap v354: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[39811]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]': finished 2024-02-14T17:48:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[39811]: osdmap e264: 8 total, 8 up, 8 in 2024-02-14T17:48:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[39811]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[39811]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi178-80647-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[39811]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi178-80647-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[39811]: pgmap v354: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[30321]: from='client.56825 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi178-80647-40"}]': finished 2024-02-14T17:48:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[30321]: osdmap e264: 8 total, 8 up, 8 in 2024-02-14T17:48:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[30321]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[30321]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi178-80647-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[30321]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi178-80647-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:00 smithi178 ceph-mon[30321]: pgmap v354: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:01.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:48:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:48:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:01 smithi184 ceph-mon[33536]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi178-80647-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:01 smithi184 ceph-mon[33536]: osdmap e265: 8 total, 8 up, 8 in 2024-02-14T17:48:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi178-80647-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:01 smithi184 ceph-mon[33536]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi178-80647-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:01 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:01 smithi184 ceph-mon[33536]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:01 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:02.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[39811]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi178-80647-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[39811]: osdmap e265: 8 total, 8 up, 8 in 2024-02-14T17:48:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi178-80647-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[39811]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi178-80647-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[39811]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[30321]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi178-80647-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[30321]: osdmap e265: 8 total, 8 up, 8 in 2024-02-14T17:48:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi178-80647-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[30321]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi178-80647-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[30321]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:02.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:01 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]': finished 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: osdmap e266: 8 total, 8 up, 8 in 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: pgmap v357: 228 pgs: 8 creating+peering, 24 unknown, 196 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi178-80647-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi178-80647-41"}]': finished 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]': finished 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: osdmap e267: 8 total, 8 up, 8 in 2024-02-14T17:48:02.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:02.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:02 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]': finished 2024-02-14T17:48:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: osdmap e266: 8 total, 8 up, 8 in 2024-02-14T17:48:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: pgmap v357: 228 pgs: 8 creating+peering, 24 unknown, 196 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi178-80647-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi178-80647-41"}]': finished 2024-02-14T17:48:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]': finished 2024-02-14T17:48:03.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:03.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: osdmap e267: 8 total, 8 up, 8 in 2024-02-14T17:48:03.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:03.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:03.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:03.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi178-80663-50"}]': finished 2024-02-14T17:48:03.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2511287242' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:03.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:03.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: osdmap e266: 8 total, 8 up, 8 in 2024-02-14T17:48:03.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]: dispatch 2024-02-14T17:48:03.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:03.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: pgmap v357: 228 pgs: 8 creating+peering, 24 unknown, 196 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:03.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi178-80647-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi178-80647-41"}]': finished 2024-02-14T17:48:03.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi178-80663-50"}]': finished 2024-02-14T17:48:03.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:03.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: osdmap e267: 8 total, 8 up, 8 in 2024-02-14T17:48:03.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:03.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:02 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:03 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:03 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]': finished 2024-02-14T17:48:03.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:03 smithi184 ceph-mon[33536]: osdmap e268: 8 total, 8 up, 8 in 2024-02-14T17:48:03.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-35", "mode": "writeback"}]: dispatch 2024-02-14T17:48:03.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3461141955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:03.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:03 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-35", "mode": "writeback"}]: dispatch 2024-02-14T17:48:03.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:03 smithi184 ceph-mon[33536]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:04.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]': finished 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[39811]: osdmap e268: 8 total, 8 up, 8 in 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-35", "mode": "writeback"}]: dispatch 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3461141955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-35", "mode": "writeback"}]: dispatch 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[39811]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]': finished 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[30321]: osdmap e268: 8 total, 8 up, 8 in 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-35", "mode": "writeback"}]: dispatch 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3461141955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-35", "mode": "writeback"}]: dispatch 2024-02-14T17:48:04.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:03 smithi178 ceph-mon[30321]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:04.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:04 smithi184 ceph-mon[33536]: pgmap v360: 268 pgs: 8 creating+peering, 64 unknown, 196 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:04 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:04 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-35", "mode": "writeback"}]': finished 2024-02-14T17:48:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:04 smithi184 ceph-mon[33536]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:04 smithi184 ceph-mon[33536]: osdmap e269: 8 total, 8 up, 8 in 2024-02-14T17:48:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:04 smithi184 ceph-mon[33536]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[39811]: pgmap v360: 268 pgs: 8 creating+peering, 64 unknown, 196 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-35", "mode": "writeback"}]': finished 2024-02-14T17:48:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[39811]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[39811]: osdmap e269: 8 total, 8 up, 8 in 2024-02-14T17:48:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[39811]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[30321]: pgmap v360: 268 pgs: 8 creating+peering, 64 unknown, 196 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-35", "mode": "writeback"}]': finished 2024-02-14T17:48:05.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[30321]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi178-80663-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:05.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[30321]: osdmap e269: 8 total, 8 up, 8 in 2024-02-14T17:48:05.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:05.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:04 smithi178 ceph-mon[30321]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:06.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:48:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:48:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:06 smithi184 ceph-mon[33536]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]': finished 2024-02-14T17:48:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:06 smithi184 ceph-mon[33536]: osdmap e270: 8 total, 8 up, 8 in 2024-02-14T17:48:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:06 smithi184 ceph-mon[33536]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:06.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:06 smithi184 ceph-mon[33536]: pgmap v363: 228 pgs: 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[39811]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]': finished 2024-02-14T17:48:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[39811]: osdmap e270: 8 total, 8 up, 8 in 2024-02-14T17:48:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[39811]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[39811]: pgmap v363: 228 pgs: 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:06.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[30321]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi178-80647-41"}]': finished 2024-02-14T17:48:06.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[30321]: osdmap e270: 8 total, 8 up, 8 in 2024-02-14T17:48:06.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2380909518' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:06.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[30321]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]: dispatch 2024-02-14T17:48:06.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:06.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:06.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:06.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:06 smithi178 ceph-mon[30321]: pgmap v363: 228 pgs: 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]': finished 2024-02-14T17:48:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: osdmap e271: 8 total, 8 up, 8 in 2024-02-14T17:48:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi178-80663-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi178-80647-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi178-80647-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:07.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:07 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]': finished 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: osdmap e271: 8 total, 8 up, 8 in 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi178-80663-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi178-80647-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi178-80647-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi178-80647-41"}]': finished 2024-02-14T17:48:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: osdmap e271: 8 total, 8 up, 8 in 2024-02-14T17:48:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi178-80663-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi178-80647-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi178-80647-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:07.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:07 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:08 smithi184 ceph-mon[33536]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi178-80647-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:08 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi178-80647-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:08 smithi184 ceph-mon[33536]: osdmap e272: 8 total, 8 up, 8 in 2024-02-14T17:48:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:08 smithi184 ceph-mon[33536]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi178-80647-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:08 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:08 smithi184 ceph-mon[33536]: pgmap v366: 228 pgs: 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[39811]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi178-80647-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi178-80647-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[39811]: osdmap e272: 8 total, 8 up, 8 in 2024-02-14T17:48:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[39811]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi178-80647-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:08.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[39811]: pgmap v366: 228 pgs: 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[30321]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi178-80647-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi178-80647-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[30321]: osdmap e272: 8 total, 8 up, 8 in 2024-02-14T17:48:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[30321]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi178-80647-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:08 smithi178 ceph-mon[30321]: pgmap v366: 228 pgs: 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:48:09.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi178-80663-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]': finished 2024-02-14T17:48:09.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:09 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]': finished 2024-02-14T17:48:09.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:09 smithi184 ceph-mon[33536]: osdmap e273: 8 total, 8 up, 8 in 2024-02-14T17:48:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi178-80663-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]': finished 2024-02-14T17:48:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:09 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]': finished 2024-02-14T17:48:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:09 smithi178 ceph-mon[39811]: osdmap e273: 8 total, 8 up, 8 in 2024-02-14T17:48:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi178-80663-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]': finished 2024-02-14T17:48:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:09 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-35"}]': finished 2024-02-14T17:48:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:09 smithi178 ceph-mon[30321]: osdmap e273: 8 total, 8 up, 8 in 2024-02-14T17:48:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:10 smithi178 ceph-mon[39811]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi178-80647-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi178-80647-42"}]': finished 2024-02-14T17:48:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:10 smithi178 ceph-mon[39811]: osdmap e274: 8 total, 8 up, 8 in 2024-02-14T17:48:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:10 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:10 smithi178 ceph-mon[39811]: pgmap v369: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:10 smithi178 ceph-mon[30321]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi178-80647-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi178-80647-42"}]': finished 2024-02-14T17:48:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:10 smithi178 ceph-mon[30321]: osdmap e274: 8 total, 8 up, 8 in 2024-02-14T17:48:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:10 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:10 smithi178 ceph-mon[30321]: pgmap v369: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:10.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:10 smithi184 ceph-mon[33536]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi178-80647-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi178-80647-42"}]': finished 2024-02-14T17:48:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:10 smithi184 ceph-mon[33536]: osdmap e274: 8 total, 8 up, 8 in 2024-02-14T17:48:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:10 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:10 smithi184 ceph-mon[33536]: pgmap v369: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:11.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:48:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:48:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[39811]: osdmap e275: 8 total, 8 up, 8 in 2024-02-14T17:48:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[30321]: osdmap e275: 8 total, 8 up, 8 in 2024-02-14T17:48:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:11 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:11.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:11 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:11 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:11 smithi184 ceph-mon[33536]: osdmap e275: 8 total, 8 up, 8 in 2024-02-14T17:48:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:11 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35"}]: dispatch 2024-02-14T17:48:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:11 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35"}]': finished 2024-02-14T17:48:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]': finished 2024-02-14T17:48:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[39811]: osdmap e276: 8 total, 8 up, 8 in 2024-02-14T17:48:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[39811]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[39811]: pgmap v372: 228 pgs: 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:48:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35"}]': finished 2024-02-14T17:48:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]': finished 2024-02-14T17:48:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[30321]: osdmap e276: 8 total, 8 up, 8 in 2024-02-14T17:48:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[30321]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:12 smithi178 ceph-mon[30321]: pgmap v372: 228 pgs: 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:48:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:12 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:12 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-35"}]': finished 2024-02-14T17:48:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi178-80663-52"}]': finished 2024-02-14T17:48:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:12 smithi184 ceph-mon[33536]: osdmap e276: 8 total, 8 up, 8 in 2024-02-14T17:48:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi178-80663-52"}]: dispatch 2024-02-14T17:48:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:12 smithi184 ceph-mon[33536]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:12 smithi184 ceph-mon[33536]: pgmap v372: 228 pgs: 228 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:48:13.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7014 ms) 2024-02-14T17:48:13.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-02-14T17:48:13.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6987 ms) 2024-02-14T17:48:13.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-02-14T17:48:13.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7007 ms) 2024-02-14T17:48:13.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-02-14T17:48:13.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7026 ms) 2024-02-14T17:48:13.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-14T17:48:13.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7021 ms) 2024-02-14T17:48:13.497 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7038 ms) 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7051 ms) 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (6872 ms) 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7015 ms) 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7012 ms) 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6991 ms) 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6985 ms) 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-14T17:48:13.498 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7004 ms) 2024-02-14T17:48:13.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-02-14T17:48:13.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7033 ms) 2024-02-14T17:48:13.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-02-14T17:48:13.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7021 ms) 2024-02-14T17:48:13.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-02-14T17:48:13.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7039 ms) 2024-02-14T17:48:13.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112117 ms total) 2024-02-14T17:48:13.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: 2024-02-14T17:48:13.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [----------] Global test environment tear-down 2024-02-14T17:48:13.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (209449 ms total) 2024-02-14T17:48:13.499 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio: [ PASSED ] 42 tests. 2024-02-14T17:48:13.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi178-80663-52"}]': finished 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]': finished 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: osdmap e277: 8 total, 8 up, 8 in 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi178-80663-52"}]': finished 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]': finished 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: osdmap e277: 8 total, 8 up, 8 in 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1835289032' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi178-80663-52"}]': finished 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi178-80647-42"}]': finished 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: osdmap e277: 8 total, 8 up, 8 in 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/802159730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='client.56855 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]: dispatch 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:15.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[39811]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]': finished 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[39811]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[39811]: osdmap e278: 8 total, 8 up, 8 in 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi178-80663-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[39811]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi178-80663-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[39811]: pgmap v375: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[30321]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]': finished 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[30321]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[30321]: osdmap e278: 8 total, 8 up, 8 in 2024-02-14T17:48:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi178-80663-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:15.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[30321]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi178-80663-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:15.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[30321]: pgmap v375: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:48:15.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:14 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:15.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:14 smithi184 ceph-mon[33536]: from='client.56855 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi178-80647-42"}]': finished 2024-02-14T17:48:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:14 smithi184 ceph-mon[33536]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:14 smithi184 ceph-mon[33536]: osdmap e278: 8 total, 8 up, 8 in 2024-02-14T17:48:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi178-80663-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:14 smithi184 ceph-mon[33536]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi178-80663-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:14 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:14 smithi184 ceph-mon[33536]: pgmap v375: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:48:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:14 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:16.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:48:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[39811]: osdmap e279: 8 total, 8 up, 8 in 2024-02-14T17:48:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[39811]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi178-80663-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi178-80663-53"}]': finished 2024-02-14T17:48:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[39811]: osdmap e280: 8 total, 8 up, 8 in 2024-02-14T17:48:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:16.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:16.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:16.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[30321]: osdmap e279: 8 total, 8 up, 8 in 2024-02-14T17:48:16.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:16.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[30321]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi178-80663-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi178-80663-53"}]': finished 2024-02-14T17:48:16.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:16.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[30321]: osdmap e280: 8 total, 8 up, 8 in 2024-02-14T17:48:16.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:16.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:16.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:16.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:15 smithi184 ceph-mon[33536]: osdmap e279: 8 total, 8 up, 8 in 2024-02-14T17:48:16.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:16.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:15 smithi184 ceph-mon[33536]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi178-80663-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi178-80663-53"}]': finished 2024-02-14T17:48:16.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:16.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:15 smithi184 ceph-mon[33536]: osdmap e280: 8 total, 8 up, 8 in 2024-02-14T17:48:16.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:16.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:17.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:16 smithi178 ceph-mon[39811]: pgmap v378: 236 pgs: 7 creating+peering, 1 unknown, 228 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-14T17:48:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-37"}]': finished 2024-02-14T17:48:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:16 smithi178 ceph-mon[39811]: osdmap e281: 8 total, 8 up, 8 in 2024-02-14T17:48:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-37", "mode": "writeback"}]: dispatch 2024-02-14T17:48:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-37", "mode": "writeback"}]: dispatch 2024-02-14T17:48:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:16 smithi178 ceph-mon[30321]: pgmap v378: 236 pgs: 7 creating+peering, 1 unknown, 228 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-14T17:48:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-37"}]': finished 2024-02-14T17:48:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:16 smithi178 ceph-mon[30321]: osdmap e281: 8 total, 8 up, 8 in 2024-02-14T17:48:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-37", "mode": "writeback"}]: dispatch 2024-02-14T17:48:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-37", "mode": "writeback"}]: dispatch 2024-02-14T17:48:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:16 smithi184 ceph-mon[33536]: pgmap v378: 236 pgs: 7 creating+peering, 1 unknown, 228 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-14T17:48:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-37"}]': finished 2024-02-14T17:48:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:16 smithi184 ceph-mon[33536]: osdmap e281: 8 total, 8 up, 8 in 2024-02-14T17:48:17.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-37", "mode": "writeback"}]: dispatch 2024-02-14T17:48:17.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-37", "mode": "writeback"}]: dispatch 2024-02-14T17:48:18.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-37", "mode": "writeback"}]': finished 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[30321]: osdmap e282: 8 total, 8 up, 8 in 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[30321]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-37", "mode": "writeback"}]': finished 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[39811]: osdmap e282: 8 total, 8 up, 8 in 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[39811]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:17 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:18.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:17 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:17 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-37", "mode": "writeback"}]': finished 2024-02-14T17:48:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:17 smithi184 ceph-mon[33536]: osdmap e282: 8 total, 8 up, 8 in 2024-02-14T17:48:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:17 smithi184 ceph-mon[33536]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:18.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:17 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[39811]: pgmap v381: 228 pgs: 228 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[39811]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]': finished 2024-02-14T17:48:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[39811]: osdmap e283: 8 total, 8 up, 8 in 2024-02-14T17:48:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[39811]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:19.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[30321]: pgmap v381: 228 pgs: 228 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:19.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[30321]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]': finished 2024-02-14T17:48:19.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:19.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[30321]: osdmap e283: 8 total, 8 up, 8 in 2024-02-14T17:48:19.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:19.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:19.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:19.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:18 smithi178 ceph-mon[30321]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:19.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:18 smithi184 ceph-mon[33536]: pgmap v381: 228 pgs: 228 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:18 smithi184 ceph-mon[33536]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi178-80663-53"}]': finished 2024-02-14T17:48:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:18 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:18 smithi184 ceph-mon[33536]: osdmap e283: 8 total, 8 up, 8 in 2024-02-14T17:48:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3949653600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:18 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37"}]: dispatch 2024-02-14T17:48:19.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:18 smithi184 ceph-mon[33536]: from='client.56861 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]: dispatch 2024-02-14T17:48:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37"}]': finished 2024-02-14T17:48:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[30321]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]': finished 2024-02-14T17:48:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[30321]: osdmap e284: 8 total, 8 up, 8 in 2024-02-14T17:48:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[30321]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[30321]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi178-80663-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[30321]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi178-80663-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:20.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:20.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:20.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37"}]': finished 2024-02-14T17:48:20.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[39811]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]': finished 2024-02-14T17:48:20.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[39811]: osdmap e284: 8 total, 8 up, 8 in 2024-02-14T17:48:20.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[39811]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[39811]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi178-80663-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:20.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:19 smithi178 ceph-mon[39811]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi178-80663-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:20.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:19 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:20.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:19 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-37"}]': finished 2024-02-14T17:48:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:19 smithi184 ceph-mon[33536]: from='client.56861 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi178-80663-53"}]': finished 2024-02-14T17:48:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:19 smithi184 ceph-mon[33536]: osdmap e284: 8 total, 8 up, 8 in 2024-02-14T17:48:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:19 smithi184 ceph-mon[33536]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:19 smithi184 ceph-mon[33536]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi178-80663-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:20.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:19 smithi184 ceph-mon[33536]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi178-80663-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:21.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:20 smithi178 ceph-mon[39811]: pgmap v384: 228 pgs: 228 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:20 smithi178 ceph-mon[39811]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi178-80663-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:20 smithi178 ceph-mon[39811]: osdmap e285: 8 total, 8 up, 8 in 2024-02-14T17:48:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi178-80663-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:20 smithi178 ceph-mon[39811]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi178-80663-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:20 smithi178 ceph-mon[30321]: pgmap v384: 228 pgs: 228 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:20 smithi178 ceph-mon[30321]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi178-80663-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:20 smithi178 ceph-mon[30321]: osdmap e285: 8 total, 8 up, 8 in 2024-02-14T17:48:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi178-80663-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:20 smithi178 ceph-mon[30321]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi178-80663-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:21.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:20 smithi184 ceph-mon[33536]: pgmap v384: 228 pgs: 228 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:21.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:20 smithi184 ceph-mon[33536]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi178-80663-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:21.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:20 smithi184 ceph-mon[33536]: osdmap e285: 8 total, 8 up, 8 in 2024-02-14T17:48:21.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi178-80663-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:21.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:20 smithi184 ceph-mon[33536]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi178-80663-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:21.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:48:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:48:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:21 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:21 smithi178 ceph-mon[30321]: osdmap e286: 8 total, 8 up, 8 in 2024-02-14T17:48:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:21 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:21 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:21 smithi178 ceph-mon[39811]: osdmap e286: 8 total, 8 up, 8 in 2024-02-14T17:48:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:21 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:21 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:21 smithi184 ceph-mon[33536]: osdmap e286: 8 total, 8 up, 8 in 2024-02-14T17:48:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:22.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:21 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[39811]: pgmap v387: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:48:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[39811]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi178-80663-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi178-80663-54"}]': finished 2024-02-14T17:48:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[39811]: osdmap e287: 8 total, 8 up, 8 in 2024-02-14T17:48:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[30321]: pgmap v387: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:48:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[30321]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi178-80663-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi178-80663-54"}]': finished 2024-02-14T17:48:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[30321]: osdmap e287: 8 total, 8 up, 8 in 2024-02-14T17:48:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:22 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:22 smithi184 ceph-mon[33536]: pgmap v387: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:48:23.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:22 smithi184 ceph-mon[33536]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi178-80663-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi178-80663-54"}]': finished 2024-02-14T17:48:23.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:22 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:23.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:22 smithi184 ceph-mon[33536]: osdmap e287: 8 total, 8 up, 8 in 2024-02-14T17:48:23.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:23.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:22 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:24.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:24 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:24.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:24 smithi184 ceph-mon[33536]: osdmap e288: 8 total, 8 up, 8 in 2024-02-14T17:48:24.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:24.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:24 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:24.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:24 smithi184 ceph-mon[33536]: pgmap v390: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:48:24.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:24 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:24 smithi178 ceph-mon[39811]: osdmap e288: 8 total, 8 up, 8 in 2024-02-14T17:48:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:24 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:24 smithi178 ceph-mon[39811]: pgmap v390: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:48:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:24 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:24 smithi178 ceph-mon[30321]: osdmap e288: 8 total, 8 up, 8 in 2024-02-14T17:48:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:24 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:24 smithi178 ceph-mon[30321]: pgmap v390: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:48:25.688 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-39"}]': finished 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[30321]: osdmap e289: 8 total, 8 up, 8 in 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-39", "mode": "writeback"}]: dispatch 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-39", "mode": "writeback"}]: dispatch 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[30321]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-39"}]': finished 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[39811]: osdmap e289: 8 total, 8 up, 8 in 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-39", "mode": "writeback"}]: dispatch 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-39", "mode": "writeback"}]: dispatch 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[39811]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:25.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-39"}]': finished 2024-02-14T17:48:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:25 smithi184 ceph-mon[33536]: osdmap e289: 8 total, 8 up, 8 in 2024-02-14T17:48:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-39", "mode": "writeback"}]: dispatch 2024-02-14T17:48:25.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-39", "mode": "writeback"}]: dispatch 2024-02-14T17:48:25.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:25.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:25 smithi184 ceph-mon[33536]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:25.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:25 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:26.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:48:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:48:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:26.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:26 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:26.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:26 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-39", "mode": "writeback"}]': finished 2024-02-14T17:48:26.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:26 smithi184 ceph-mon[33536]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]': finished 2024-02-14T17:48:26.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:26 smithi184 ceph-mon[33536]: osdmap e290: 8 total, 8 up, 8 in 2024-02-14T17:48:26.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:26.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:26 smithi184 ceph-mon[33536]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:26.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:26.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:26 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:26.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:26 smithi184 ceph-mon[33536]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 57 op/s 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-39", "mode": "writeback"}]': finished 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[30321]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]': finished 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[30321]: osdmap e290: 8 total, 8 up, 8 in 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[30321]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[30321]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 57 op/s 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-39", "mode": "writeback"}]': finished 2024-02-14T17:48:26.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[39811]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi178-80663-54"}]': finished 2024-02-14T17:48:26.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[39811]: osdmap e290: 8 total, 8 up, 8 in 2024-02-14T17:48:26.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/4093753642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:26.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[39811]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]: dispatch 2024-02-14T17:48:26.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:26.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:26.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:26 smithi178 ceph-mon[39811]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 57 op/s 2024-02-14T17:48:27.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:27 smithi184 ceph-mon[33536]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]': finished 2024-02-14T17:48:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:27 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:27 smithi184 ceph-mon[33536]: osdmap e291: 8 total, 8 up, 8 in 2024-02-14T17:48:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:27 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:27.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi178-80663-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[30321]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]': finished 2024-02-14T17:48:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[30321]: osdmap e291: 8 total, 8 up, 8 in 2024-02-14T17:48:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi178-80663-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:27.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[39811]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi178-80663-54"}]': finished 2024-02-14T17:48:27.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:48:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[39811]: osdmap e291: 8 total, 8 up, 8 in 2024-02-14T17:48:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39"}]: dispatch 2024-02-14T17:48:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:27.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi178-80663-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: seed 80903 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (316 ms) 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (13807 ms) 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (2 ms) 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (14125 ms total) 2024-02-14T17:48:28.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (9390 ms) 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (10146 ms) 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10011 ms) 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: my_snaps [3] 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: my_snaps [4,3] 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: my_snaps [5,4,3] 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: promoting some heads 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: promoting from clones for snap 6 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: promoting from clones for snap 5 2024-02-14T17:48:28.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: promoting from clones for snap 4 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: promoting from clones for snap 3 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: waiting for scrubs... 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: done waiting 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46026 ms) 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10238 ms) 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7764 ms) 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8038 ms) 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8017 ms) 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10078 ms) 2024-02-14T17:48:28.675 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8869 ms) 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10025 ms) 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (13256 ms) 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7723 ms) 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8036 ms) 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13054 ms) 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-02-14T17:48:28.676 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7923 ms) 2024-02-14T17:48:28.677 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-02-14T17:48:28.677 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8265 ms) 2024-02-14T17:48:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:28 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:28 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39"}]': finished 2024-02-14T17:48:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi178-80663-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:28 smithi184 ceph-mon[33536]: osdmap e292: 8 total, 8 up, 8 in 2024-02-14T17:48:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi178-80663-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:28 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:28 smithi184 ceph-mon[33536]: pgmap v396: 228 pgs: 228 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 57 op/s 2024-02-14T17:48:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39"}]': finished 2024-02-14T17:48:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi178-80663-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[30321]: osdmap e292: 8 total, 8 up, 8 in 2024-02-14T17:48:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi178-80663-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[30321]: pgmap v396: 228 pgs: 228 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 57 op/s 2024-02-14T17:48:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-39"}]': finished 2024-02-14T17:48:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi178-80663-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[39811]: osdmap e292: 8 total, 8 up, 8 in 2024-02-14T17:48:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi178-80663-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:29.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:28 smithi178 ceph-mon[39811]: pgmap v396: 228 pgs: 228 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 57 op/s 2024-02-14T17:48:29.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:29 smithi184 ceph-mon[33536]: osdmap e293: 8 total, 8 up, 8 in 2024-02-14T17:48:29.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi178-80663-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi178-80663-55"}]': finished 2024-02-14T17:48:29.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:29 smithi184 ceph-mon[33536]: osdmap e294: 8 total, 8 up, 8 in 2024-02-14T17:48:29.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:29.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:29 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:29 smithi178 ceph-mon[39811]: osdmap e293: 8 total, 8 up, 8 in 2024-02-14T17:48:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi178-80663-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi178-80663-55"}]': finished 2024-02-14T17:48:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:29 smithi178 ceph-mon[39811]: osdmap e294: 8 total, 8 up, 8 in 2024-02-14T17:48:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:29 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:29 smithi178 ceph-mon[30321]: osdmap e293: 8 total, 8 up, 8 in 2024-02-14T17:48:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi178-80663-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi178-80663-55"}]': finished 2024-02-14T17:48:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:29 smithi178 ceph-mon[30321]: osdmap e294: 8 total, 8 up, 8 in 2024-02-14T17:48:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:29 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:30 smithi184 ceph-mon[33536]: pgmap v399: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:30 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:30 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:30 smithi184 ceph-mon[33536]: osdmap e295: 8 total, 8 up, 8 in 2024-02-14T17:48:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:30 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:31.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[30321]: pgmap v399: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[30321]: osdmap e295: 8 total, 8 up, 8 in 2024-02-14T17:48:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[39811]: pgmap v399: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[39811]: osdmap e295: 8 total, 8 up, 8 in 2024-02-14T17:48:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:30 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:31.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:48:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:48:31.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:31 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:31.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:31 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:31.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:31 smithi184 ceph-mon[33536]: osdmap e296: 8 total, 8 up, 8 in 2024-02-14T17:48:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:48:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:31 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:48:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:31 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:48:32.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[39811]: osdmap e296: 8 total, 8 up, 8 in 2024-02-14T17:48:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:48:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:48:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:48:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[30321]: osdmap e296: 8 total, 8 up, 8 in 2024-02-14T17:48:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:48:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:48:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:31 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:48:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: pgmap v402: 228 pgs: 4 creating+activating, 21 creating+peering, 203 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:48:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi178-80663-55"}]': finished 2024-02-14T17:48:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: osdmap e297: 8 total, 8 up, 8 in 2024-02-14T17:48:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:48:33.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: pgmap v402: 228 pgs: 4 creating+activating, 21 creating+peering, 203 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi178-80663-55"}]': finished 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: osdmap e297: 8 total, 8 up, 8 in 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:48:33.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:33 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:33.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: pgmap v402: 228 pgs: 4 creating+activating, 21 creating+peering, 203 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:48:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi178-80663-55"}]': finished 2024-02-14T17:48:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: osdmap e297: 8 total, 8 up, 8 in 2024-02-14T17:48:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi178-80663-55"}]: dispatch 2024-02-14T17:48:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:48:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:48:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:33 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:48:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi178-80663-55"}]': finished 2024-02-14T17:48:34.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:34 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:48:34.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:34 smithi184 ceph-mon[33536]: osdmap e298: 8 total, 8 up, 8 in 2024-02-14T17:48:34.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:48:34.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:34 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:48:34.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:34.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:34.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi178-80663-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:34.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:34 smithi184 ceph-mon[33536]: pgmap v405: 228 pgs: 4 creating+activating, 21 creating+peering, 203 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-14T17:48:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi178-80663-55"}]': finished 2024-02-14T17:48:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:48:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[39811]: osdmap e298: 8 total, 8 up, 8 in 2024-02-14T17:48:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:48:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:48:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi178-80663-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[39811]: pgmap v405: 228 pgs: 4 creating+activating, 21 creating+peering, 203 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-14T17:48:34.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/836489281' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi178-80663-55"}]': finished 2024-02-14T17:48:34.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:48:34.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[30321]: osdmap e298: 8 total, 8 up, 8 in 2024-02-14T17:48:34.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:48:34.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:48:34.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:34.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:34.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi178-80663-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:34.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:34 smithi178 ceph-mon[30321]: pgmap v405: 228 pgs: 4 creating+activating, 21 creating+peering, 203 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-14T17:48:35.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:35 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:48:35.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi178-80663-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:35.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:35 smithi184 ceph-mon[33536]: osdmap e299: 8 total, 8 up, 8 in 2024-02-14T17:48:35.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi178-80663-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:48:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:48:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:36.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:35 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:48:36.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi178-80663-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:36.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:35 smithi178 ceph-mon[30321]: osdmap e299: 8 total, 8 up, 8 in 2024-02-14T17:48:36.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi178-80663-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:35 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:48:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi178-80663-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:35 smithi178 ceph-mon[39811]: osdmap e299: 8 total, 8 up, 8 in 2024-02-14T17:48:36.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi178-80663-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:36.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:36.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:36 smithi184 ceph-mon[33536]: osdmap e300: 8 total, 8 up, 8 in 2024-02-14T17:48:36.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:36.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41"}]: dispatch 2024-02-14T17:48:36.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41"}]: dispatch 2024-02-14T17:48:36.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:36 smithi184 ceph-mon[33536]: pgmap v408: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:36.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi178-80663-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi178-80663-56"}]': finished 2024-02-14T17:48:36.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41"}]': finished 2024-02-14T17:48:36.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:36 smithi184 ceph-mon[33536]: osdmap e301: 8 total, 8 up, 8 in 2024-02-14T17:48:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[39811]: osdmap e300: 8 total, 8 up, 8 in 2024-02-14T17:48:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41"}]: dispatch 2024-02-14T17:48:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41"}]: dispatch 2024-02-14T17:48:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[39811]: pgmap v408: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi178-80663-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi178-80663-56"}]': finished 2024-02-14T17:48:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41"}]': finished 2024-02-14T17:48:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[39811]: osdmap e301: 8 total, 8 up, 8 in 2024-02-14T17:48:37.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[30321]: osdmap e300: 8 total, 8 up, 8 in 2024-02-14T17:48:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41"}]: dispatch 2024-02-14T17:48:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41"}]: dispatch 2024-02-14T17:48:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[30321]: pgmap v408: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi178-80663-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi178-80663-56"}]': finished 2024-02-14T17:48:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-41"}]': finished 2024-02-14T17:48:37.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:36 smithi178 ceph-mon[30321]: osdmap e301: 8 total, 8 up, 8 in 2024-02-14T17:48:38.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:38 smithi184 ceph-mon[33536]: osdmap e302: 8 total, 8 up, 8 in 2024-02-14T17:48:38.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:38 smithi184 ceph-mon[33536]: pgmap v411: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:38.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:38 smithi178 ceph-mon[39811]: osdmap e302: 8 total, 8 up, 8 in 2024-02-14T17:48:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:38 smithi178 ceph-mon[39811]: pgmap v411: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:38 smithi178 ceph-mon[30321]: osdmap e302: 8 total, 8 up, 8 in 2024-02-14T17:48:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:38 smithi178 ceph-mon[30321]: pgmap v411: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:39.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:39 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:39.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:39 smithi184 ceph-mon[33536]: osdmap e303: 8 total, 8 up, 8 in 2024-02-14T17:48:39.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:39.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:39.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:39 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:39 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:39 smithi178 ceph-mon[39811]: osdmap e303: 8 total, 8 up, 8 in 2024-02-14T17:48:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:39 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:39 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:39 smithi178 ceph-mon[30321]: osdmap e303: 8 total, 8 up, 8 in 2024-02-14T17:48:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:39 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi178-80663-56"}]': finished 2024-02-14T17:48:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[39811]: osdmap e304: 8 total, 8 up, 8 in 2024-02-14T17:48:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi178-80903-6","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:48:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[39811]: pgmap v414: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi178-80663-56"}]': finished 2024-02-14T17:48:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[30321]: osdmap e304: 8 total, 8 up, 8 in 2024-02-14T17:48:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi178-80903-6","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:48:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:40 smithi178 ceph-mon[30321]: pgmap v414: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi178-80663-56"}]': finished 2024-02-14T17:48:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:40 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:40 smithi184 ceph-mon[33536]: osdmap e304: 8 total, 8 up, 8 in 2024-02-14T17:48:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi178-80663-56"}]: dispatch 2024-02-14T17:48:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi178-80903-6","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:48:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:40 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:40 smithi184 ceph-mon[33536]: pgmap v414: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:41.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:48:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:48:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi178-80663-56"}]': finished 2024-02-14T17:48:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[39811]: osdmap e305: 8 total, 8 up, 8 in 2024-02-14T17:48:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:48:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:48:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi178-80663-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi178-80663-56"}]': finished 2024-02-14T17:48:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[30321]: osdmap e305: 8 total, 8 up, 8 in 2024-02-14T17:48:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:48:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:48:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi178-80663-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:41 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:41.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/526419956' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi178-80663-56"}]': finished 2024-02-14T17:48:41.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:41 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:41.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:41 smithi184 ceph-mon[33536]: osdmap e305: 8 total, 8 up, 8 in 2024-02-14T17:48:41.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:48:41.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:41 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:48:41.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:41.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:41.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi178-80663-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:41.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:41 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_count","val": "8"}]': finished 2024-02-14T17:48:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi178-80663-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[39811]: osdmap e306: 8 total, 8 up, 8 in 2024-02-14T17:48:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi178-80663-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[39811]: pgmap v417: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:43.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:48:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_count","val": "8"}]': finished 2024-02-14T17:48:43.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi178-80663-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:43.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[30321]: osdmap e306: 8 total, 8 up, 8 in 2024-02-14T17:48:43.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:43.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi178-80663-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:43.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:43.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[30321]: pgmap v417: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:43.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:42 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:48:43.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:42 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_count","val": "8"}]': finished 2024-02-14T17:48:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi178-80663-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:42 smithi184 ceph-mon[33536]: osdmap e306: 8 total, 8 up, 8 in 2024-02-14T17:48:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi178-80663-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:42 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:48:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:42 smithi184 ceph-mon[33536]: pgmap v417: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:42 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:48:43.693 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-02-14T17:48:43.693 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-14T17:48:43.693 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: ok, hit_set contains 275:602f83fe:::foo:head 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (8782 ms) 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg_num = 32 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 0 ls 1707932924,0 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 1 ls 1707932924,0 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 2 ls 1707932924,0 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 3 ls 1707932924,0 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 4 ls 1707932924,0 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 5 ls 1707932924,0 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 6 ls 1707932924,0 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 7 ls 1707932924,0 2024-02-14T17:48:43.694 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 8 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 9 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 10 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 11 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 12 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 13 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 14 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 15 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 16 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 17 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 18 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 19 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 20 ls 1707932924,0 2024-02-14T17:48:43.695 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 21 ls 1707932924,0 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 22 ls 1707932924,0 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 23 ls 1707932924,0 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 24 ls 1707932924,0 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 25 ls 1707932924,0 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 26 ls 1707932924,0 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 27 ls 1707932924,0 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 28 ls 1707932924,0 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 29 ls 1707932924,0 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 30 ls 1707932924,0 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg 31 ls 1707932924,0 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: pg_num = 32 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:6cac518f:::0:head 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:02547ec2:::1:head 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:f905c69b:::2:head 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:cfc208b3:::3:head 2024-02-14T17:48:43.696 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:d83876eb:::4:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:b29083e3:::5:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:c4fdafeb:::6:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:5c6b0b28:::7:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:bd63b0f1:::8:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:e960b815:::9:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:52ea6a34:::10:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:89d3ae78:::11:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:de5d7c5f:::12:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:566253c9:::13:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:62a1935d:::14:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:863748b0:::15:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:3958e169:::16:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:4d4dabf9:::17:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:8391935d:::18:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:28883081:::19:head 2024-02-14T17:48:43.697 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:69259c59:::20:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:4bdb80b7:::21:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:a11c5d71:::22:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:271af37b:::23:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:95b121be:::24:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:58d1031b:::25:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:0a050783:::26:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:c709704c:::27:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:cbe56eaf:::28:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:86b4b162:::29:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:70d89383:::30:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:dd450c7c:::31:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:6d5729b1:::32:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:c388f3fb:::33:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:56cfea31:::34:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:9dbc1bf7:::35:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:40b74ccd:::36:head 2024-02-14T17:48:43.698 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:4d5aaf42:::37:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:920f362c:::38:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:6cc53222:::39:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:9cad833f:::40:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:1ea84d41:::41:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:c4480ef6:::42:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:a694361e:::43:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:d1bd33e9:::44:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:ddc2cd5d:::45:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:2b782207:::46:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:7b187fca:::47:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:90ecdf6f:::48:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:a5ed95fe:::49:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:ea0eaa55:::50:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:f33ef17b:::51:head 2024-02-14T17:48:43.699 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:a0d1b2f6:::52:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:60c5229e:::53:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:edcbc575:::54:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:102cf253:::55:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:efb7fb0b:::56:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:50d0a326:::57:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:d4dc5daf:::58:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:3a130462:::59:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:ec87ed71:::60:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:d5bc9454:::61:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:3ddfe313:::62:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:7c2816b9:::63:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:47e00e4d:::64:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:c6410c18:::65:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:b48ed237:::66:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:cd63ad31:::67:head 2024-02-14T17:48:43.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:b179e92b:::68:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:0d9f741a:::69:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:6d3352ae:::70:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:c6d5c19e:::71:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:bc4729c3:::72:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:77e930b9:::73:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:0abeecfd:::74:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:b7c37e15:::75:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:b6378398:::76:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:02bd68de:::77:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:cc795d2d:::78:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:630d4fea:::79:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:e0d29ef5:::80:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:fd6f13d2:::81:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:606461d5:::82:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:eadbdc43:::83:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:8761d0bb:::84:head 2024-02-14T17:48:43.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:9ef0186f:::85:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:e0d41294:::86:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:961de695:::87:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:1423148f:::88:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:633a8fa2:::89:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:a8653809:::90:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:3dac8b33:::91:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:35aad435:::92:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:f6dcc343:::93:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:dbbdad87:::94:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:1cb48ce0:::95:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:03cd461c:::96:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:17a4ea99:::97:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:9993c9a7:::98:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:6394211c:::99:head 2024-02-14T17:48:43.702 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:94c7ae57:::100:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:6fdee5bb:::101:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:9a477fd1:::102:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:eb850916:::103:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:affc56b9:::104:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:b42dc814:::105:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:f319f8f0:::106:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:9a40b9de:::107:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:8b524f28:::108:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:e3de589f:::109:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:90f90a5b:::110:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:a7b4f1d7:::111:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:af51766e:::112:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:b6f90bd1:::113:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:e0261208:::114:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:c9569ef7:::115:head 2024-02-14T17:48:43.703 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:61bebe50:::116:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:fe93412b:::117:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:d3d38bee:::118:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:3100ba0c:::119:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:d0560ada:::120:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:f0ea8b35:::121:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:766f231a:::122:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:a07a2582:::123:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:bd7c6b3a:::124:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:fb2ddaff:::125:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:4408e1fe:::126:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:ee1df7a7:::127:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:c3002909:::128:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:4f48ffa9:::129:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:edf38733:::130:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:c08425c0:::131:head 2024-02-14T17:48:43.704 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:5f902d98:::132:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:41ea2c93:::133:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:813cee13:::134:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:0131818d:::135:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:26ba5a85:::136:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:381b8a5a:::137:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:28797e47:::138:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:bfca7f22:::139:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:36807075:::140:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:80b03975:::141:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:5c15709b:::142:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:f39ea15e:::143:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:ea992956:::144:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:48887b1c:::145:head 2024-02-14T17:48:43.705 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:9f24a9dd:::146:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:987f100b:::147:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:d2dd3581:::148:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:7fed1808:::149:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:c80b70e9:::150:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:85ed90f9:::151:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:36428b24:::152:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:d044c34a:::153:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:7c18bf58:::154:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:d1c21232:::155:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:a7a3c575:::156:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:87da0633:::157:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:d5ac3822:::158:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:3f20522d:::159:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:6ca26563:::160:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:532ce135:::161:head 2024-02-14T17:48:43.706 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:c78863e6:::162:head 2024-02-14T17:48:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:48:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[39811]: osdmap e307: 8 total, 8 up, 8 in 2024-02-14T17:48:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:48:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi178-80663-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi178-80663-57"}]': finished 2024-02-14T17:48:44.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-14T17:48:44.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[39811]: osdmap e308: 8 total, 8 up, 8 in 2024-02-14T17:48:44.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi178-80903-43","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:48:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:48:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[30321]: osdmap e307: 8 total, 8 up, 8 in 2024-02-14T17:48:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:48:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi178-80663-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi178-80663-57"}]': finished 2024-02-14T17:48:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-14T17:48:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[30321]: osdmap e308: 8 total, 8 up, 8 in 2024-02-14T17:48:44.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi178-80903-43","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:48:44.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:48:44.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:43 smithi184 ceph-mon[33536]: osdmap e307: 8 total, 8 up, 8 in 2024-02-14T17:48:44.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:48:44.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:44.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi178-80663-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi178-80663-57"}]': finished 2024-02-14T17:48:44.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-14T17:48:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:43 smithi184 ceph-mon[33536]: osdmap e308: 8 total, 8 up, 8 in 2024-02-14T17:48:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi178-80903-43","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:48:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43"}]: dispatch 2024-02-14T17:48:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43"}]: dispatch 2024-02-14T17:48:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[39811]: pgmap v420: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43"}]': finished 2024-02-14T17:48:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[39811]: osdmap e309: 8 total, 8 up, 8 in 2024-02-14T17:48:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43"}]: dispatch 2024-02-14T17:48:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43"}]: dispatch 2024-02-14T17:48:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[30321]: pgmap v420: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43"}]': finished 2024-02-14T17:48:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:44 smithi178 ceph-mon[30321]: osdmap e309: 8 total, 8 up, 8 in 2024-02-14T17:48:45.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:44 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:48:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43"}]: dispatch 2024-02-14T17:48:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:44 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43"}]: dispatch 2024-02-14T17:48:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:44 smithi184 ceph-mon[33536]: pgmap v420: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:44 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-43"}]': finished 2024-02-14T17:48:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:44 smithi184 ceph-mon[33536]: osdmap e309: 8 total, 8 up, 8 in 2024-02-14T17:48:45.984 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:48:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:48:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:45 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:45 smithi178 ceph-mon[30321]: osdmap e310: 8 total, 8 up, 8 in 2024-02-14T17:48:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:45.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:45 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:45.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:45 smithi178 ceph-mon[39811]: osdmap e310: 8 total, 8 up, 8 in 2024-02-14T17:48:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:46.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:45 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:46.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:45 smithi184 ceph-mon[33536]: osdmap e310: 8 total, 8 up, 8 in 2024-02-14T17:48:46.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[30321]: pgmap v423: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi178-80663-57"}]': finished 2024-02-14T17:48:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[30321]: osdmap e311: 8 total, 8 up, 8 in 2024-02-14T17:48:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[39811]: pgmap v423: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi178-80663-57"}]': finished 2024-02-14T17:48:47.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[39811]: osdmap e311: 8 total, 8 up, 8 in 2024-02-14T17:48:47.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:47.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:47.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:46 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:47.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:46 smithi184 ceph-mon[33536]: pgmap v423: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:47.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi178-80663-57"}]': finished 2024-02-14T17:48:47.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:46 smithi184 ceph-mon[33536]: osdmap e311: 8 total, 8 up, 8 in 2024-02-14T17:48:47.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi178-80663-57"}]: dispatch 2024-02-14T17:48:47.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:47.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:46 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi178-80663-57"}]': finished 2024-02-14T17:48:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: osdmap e312: 8 total, 8 up, 8 in 2024-02-14T17:48:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi178-80663-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:48.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi178-80663-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:48.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[30321]: pgmap v426: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:48.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi178-80663-57"}]': finished 2024-02-14T17:48:48.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:48.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: osdmap e312: 8 total, 8 up, 8 in 2024-02-14T17:48:48.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi178-80663-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:48.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi178-80663-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:48.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:48 smithi178 ceph-mon[39811]: pgmap v426: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2176839615' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi178-80663-57"}]': finished 2024-02-14T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: osdmap e312: 8 total, 8 up, 8 in 2024-02-14T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi178-80663-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi178-80663-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:48 smithi184 ceph-mon[33536]: pgmap v426: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:50.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:50.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[39811]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi178-80663-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:50.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[39811]: osdmap e313: 8 total, 8 up, 8 in 2024-02-14T17:48:50.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi178-80663-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:50.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[39811]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi178-80663-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:50.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:48:50.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:48:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[30321]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi178-80663-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[30321]: osdmap e313: 8 total, 8 up, 8 in 2024-02-14T17:48:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi178-80663-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[30321]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi178-80663-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:50.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:48:50.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:49 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:48:50.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:49 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:50.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:49 smithi184 ceph-mon[33536]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi178-80663-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:49 smithi184 ceph-mon[33536]: osdmap e313: 8 total, 8 up, 8 in 2024-02-14T17:48:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi178-80663-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:49 smithi184 ceph-mon[33536]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi178-80663-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:48:50.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:49 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:48:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_count","val": "3"}]': finished 2024-02-14T17:48:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:48:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[30321]: osdmap e314: 8 total, 8 up, 8 in 2024-02-14T17:48:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:48:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[30321]: pgmap v429: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[30321]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi178-80663-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi178-80663-58"}]': finished 2024-02-14T17:48:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_period","val": "3"}]': finished 2024-02-14T17:48:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[30321]: osdmap e315: 8 total, 8 up, 8 in 2024-02-14T17:48:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:48:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:48:51.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_count","val": "3"}]': finished 2024-02-14T17:48:51.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:48:51.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[39811]: osdmap e314: 8 total, 8 up, 8 in 2024-02-14T17:48:51.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:48:51.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[39811]: pgmap v429: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:51.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:51.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[39811]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi178-80663-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi178-80663-58"}]': finished 2024-02-14T17:48:51.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_period","val": "3"}]': finished 2024-02-14T17:48:51.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[39811]: osdmap e315: 8 total, 8 up, 8 in 2024-02-14T17:48:51.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:48:51.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:50 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:48:51.150 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:50 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_count","val": "3"}]': finished 2024-02-14T17:48:51.150 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:48:51.150 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:50 smithi184 ceph-mon[33536]: osdmap e314: 8 total, 8 up, 8 in 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:50 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:50 smithi184 ceph-mon[33536]: pgmap v429: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:50 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:50 smithi184 ceph-mon[33536]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi178-80663-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi178-80663-58"}]': finished 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:50 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_period","val": "3"}]': finished 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:50 smithi184 ceph-mon[33536]: osdmap e315: 8 total, 8 up, 8 in 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:50 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:48:51.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:48:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:48:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:51 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:51 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:48:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:51 smithi178 ceph-mon[39811]: osdmap e316: 8 total, 8 up, 8 in 2024-02-14T17:48:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:48:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:51 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:48:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:51 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:51 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:48:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:51 smithi178 ceph-mon[30321]: osdmap e316: 8 total, 8 up, 8 in 2024-02-14T17:48:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:48:52.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:51 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:48:52.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:51 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:52.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:51 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:48:52.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:51 smithi184 ceph-mon[33536]: osdmap e316: 8 total, 8 up, 8 in 2024-02-14T17:48:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:48:52.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:51 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:48:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:52 smithi178 ceph-mon[39811]: pgmap v432: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:52 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T17:48:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:52 smithi178 ceph-mon[39811]: osdmap e317: 8 total, 8 up, 8 in 2024-02-14T17:48:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:52 smithi178 ceph-mon[39811]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:52 smithi178 ceph-mon[30321]: pgmap v432: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:52 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T17:48:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:52 smithi178 ceph-mon[30321]: osdmap e317: 8 total, 8 up, 8 in 2024-02-14T17:48:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:52 smithi178 ceph-mon[30321]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:52 smithi184 ceph-mon[33536]: pgmap v432: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:52 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T17:48:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:52 smithi184 ceph-mon[33536]: osdmap e317: 8 total, 8 up, 8 in 2024-02-14T17:48:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:53.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:52 smithi184 ceph-mon[33536]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:54.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:54 smithi178 ceph-mon[39811]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]': finished 2024-02-14T17:48:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:54 smithi178 ceph-mon[39811]: osdmap e318: 8 total, 8 up, 8 in 2024-02-14T17:48:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:54 smithi178 ceph-mon[39811]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:54 smithi178 ceph-mon[39811]: pgmap v435: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:54 smithi178 ceph-mon[30321]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]': finished 2024-02-14T17:48:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:54 smithi178 ceph-mon[30321]: osdmap e318: 8 total, 8 up, 8 in 2024-02-14T17:48:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:54 smithi178 ceph-mon[30321]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:54 smithi178 ceph-mon[30321]: pgmap v435: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:54 smithi184 ceph-mon[33536]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi178-80663-58"}]': finished 2024-02-14T17:48:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:54 smithi184 ceph-mon[33536]: osdmap e318: 8 total, 8 up, 8 in 2024-02-14T17:48:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1864276256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:54 smithi184 ceph-mon[33536]: from='client.49342 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]: dispatch 2024-02-14T17:48:54.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:54 smithi184 ceph-mon[33536]: pgmap v435: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:48:55.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:48:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[39811]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]': finished 2024-02-14T17:48:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[39811]: osdmap e319: 8 total, 8 up, 8 in 2024-02-14T17:48:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[39811]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[39811]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[39811]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[30321]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]': finished 2024-02-14T17:48:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[30321]: osdmap e319: 8 total, 8 up, 8 in 2024-02-14T17:48:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[30321]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[30321]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:55 smithi178 ceph-mon[30321]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:55 smithi184 ceph-mon[33536]: from='client.49342 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi178-80663-58"}]': finished 2024-02-14T17:48:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:55 smithi184 ceph-mon[33536]: osdmap e319: 8 total, 8 up, 8 in 2024-02-14T17:48:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:55 smithi184 ceph-mon[33536]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:55 smithi184 ceph-mon[33536]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:55 smithi184 ceph-mon[33536]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:56 smithi178 ceph-mon[39811]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:56 smithi178 ceph-mon[39811]: osdmap e320: 8 total, 8 up, 8 in 2024-02-14T17:48:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi178-80663-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:56 smithi178 ceph-mon[39811]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi178-80663-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:56 smithi178 ceph-mon[39811]: pgmap v438: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-14T17:48:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:56 smithi178 ceph-mon[30321]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:56 smithi178 ceph-mon[30321]: osdmap e320: 8 total, 8 up, 8 in 2024-02-14T17:48:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi178-80663-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:56 smithi178 ceph-mon[30321]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi178-80663-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:57.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:56 smithi178 ceph-mon[30321]: pgmap v438: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-14T17:48:57.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:56 smithi184 ceph-mon[33536]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:57.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:56 smithi184 ceph-mon[33536]: osdmap e320: 8 total, 8 up, 8 in 2024-02-14T17:48:57.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi178-80663-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:57.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:56 smithi184 ceph-mon[33536]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi178-80663-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:48:57.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:56 smithi184 ceph-mon[33536]: pgmap v438: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-14T17:48:58.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:57 smithi178 ceph-mon[39811]: osdmap e321: 8 total, 8 up, 8 in 2024-02-14T17:48:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:57 smithi178 ceph-mon[39811]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi178-80663-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]': finished 2024-02-14T17:48:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:57 smithi178 ceph-mon[39811]: osdmap e322: 8 total, 8 up, 8 in 2024-02-14T17:48:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:57 smithi178 ceph-mon[30321]: osdmap e321: 8 total, 8 up, 8 in 2024-02-14T17:48:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:57 smithi178 ceph-mon[30321]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi178-80663-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]': finished 2024-02-14T17:48:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:57 smithi178 ceph-mon[30321]: osdmap e322: 8 total, 8 up, 8 in 2024-02-14T17:48:58.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:57 smithi184 ceph-mon[33536]: osdmap e321: 8 total, 8 up, 8 in 2024-02-14T17:48:58.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:57 smithi184 ceph-mon[33536]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi178-80663-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]': finished 2024-02-14T17:48:58.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:57 smithi184 ceph-mon[33536]: osdmap e322: 8 total, 8 up, 8 in 2024-02-14T17:48:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:58 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:58 smithi178 ceph-mon[39811]: pgmap v441: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-14T17:48:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:58 smithi178 ceph-mon[39811]: osdmap e323: 8 total, 8 up, 8 in 2024-02-14T17:48:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:58 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:58 smithi178 ceph-mon[30321]: pgmap v441: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-14T17:48:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:58 smithi178 ceph-mon[30321]: osdmap e323: 8 total, 8 up, 8 in 2024-02-14T17:48:59.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:58 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:58 smithi184 ceph-mon[33536]: pgmap v441: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-14T17:48:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:58 smithi184 ceph-mon[33536]: osdmap e323: 8 total, 8 up, 8 in 2024-02-14T17:49:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:59 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:59 smithi178 ceph-mon[30321]: osdmap e324: 8 total, 8 up, 8 in 2024-02-14T17:49:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:48:59 smithi178 ceph-mon[30321]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:59 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:59 smithi178 ceph-mon[39811]: osdmap e324: 8 total, 8 up, 8 in 2024-02-14T17:49:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:48:59 smithi178 ceph-mon[39811]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:00.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:59 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:00.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:59 smithi184 ceph-mon[33536]: osdmap e324: 8 total, 8 up, 8 in 2024-02-14T17:49:00.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:00.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:48:59 smithi184 ceph-mon[33536]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:00 smithi178 ceph-mon[39811]: pgmap v444: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:00 smithi178 ceph-mon[39811]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]': finished 2024-02-14T17:49:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:00 smithi178 ceph-mon[39811]: osdmap e325: 8 total, 8 up, 8 in 2024-02-14T17:49:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:00 smithi178 ceph-mon[39811]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:00 smithi178 ceph-mon[30321]: pgmap v444: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:00 smithi178 ceph-mon[30321]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]': finished 2024-02-14T17:49:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:00 smithi178 ceph-mon[30321]: osdmap e325: 8 total, 8 up, 8 in 2024-02-14T17:49:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:01.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:00 smithi178 ceph-mon[30321]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:01.161 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:00 smithi184 ceph-mon[33536]: pgmap v444: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:01.161 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:00 smithi184 ceph-mon[33536]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi178-80663-59"}]': finished 2024-02-14T17:49:01.162 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:00 smithi184 ceph-mon[33536]: osdmap e325: 8 total, 8 up, 8 in 2024-02-14T17:49:01.162 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1431556177' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:01.162 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:00 smithi184 ceph-mon[33536]: from='client.56897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]: dispatch 2024-02-14T17:49:01.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:49:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:49:01.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:7462ddf6::o.RoundTripAppendPP (3007 ms) 2024-02-14T17:49:01.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-02-14T17:49:01.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (2996 ms) 2024-02-14T17:49:01.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-14T17:49:01.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3246 ms) 2024-02-14T17:49:01.524 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3009 ms) 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: setting pool EIO 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: max_success 101, min_failed 102 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4022 ms) 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (2771 ms) 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (130381 ms total) 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3007 ms) 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9067 ms) 2024-02-14T17:49:01.525 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14894 ms) 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3010 ms) 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29978 ms total) 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3005 ms) 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3005 ms total) 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (14017 ms) 2024-02-14T17:49:01.526 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6970 ms) 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7014 ms) 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7030 ms) 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3006 ms) 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7039 ms) 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6920 ms) 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7025 ms) 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-02-14T17:49:01.527 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7011 ms) 2024-02-14T17:49:01.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-14T17:49:01.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7030 ms) 2024-02-14T17:49:01.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-02-14T17:49:01.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7015 ms) 2024-02-14T17:49:01.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-14T17:49:01.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7017 ms) 2024-02-14T17:49:01.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-14T17:49:01.528 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7017 ms) 2024-02-14T17:49:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:01 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:01 smithi178 ceph-mon[39811]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]': finished 2024-02-14T17:49:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:01 smithi178 ceph-mon[39811]: osdmap e326: 8 total, 8 up, 8 in 2024-02-14T17:49:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:01 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:01 smithi178 ceph-mon[30321]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]': finished 2024-02-14T17:49:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:01 smithi178 ceph-mon[30321]: osdmap e326: 8 total, 8 up, 8 in 2024-02-14T17:49:02.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:01 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:02.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:01 smithi184 ceph-mon[33536]: from='client.56897 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi178-80663-59"}]': finished 2024-02-14T17:49:02.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:01 smithi184 ceph-mon[33536]: osdmap e326: 8 total, 8 up, 8 in 2024-02-14T17:49:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:02 smithi178 ceph-mon[30321]: pgmap v447: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:49:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:02 smithi178 ceph-mon[30321]: osdmap e327: 8 total, 8 up, 8 in 2024-02-14T17:49:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/827031536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:02 smithi178 ceph-mon[30321]: from='client.56903 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:02 smithi178 ceph-mon[39811]: pgmap v447: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:49:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:02 smithi178 ceph-mon[39811]: osdmap e327: 8 total, 8 up, 8 in 2024-02-14T17:49:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/827031536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:02 smithi178 ceph-mon[39811]: from='client.56903 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:03.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:02 smithi184 ceph-mon[33536]: pgmap v447: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:49:03.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:02 smithi184 ceph-mon[33536]: osdmap e327: 8 total, 8 up, 8 in 2024-02-14T17:49:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/827031536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:02 smithi184 ceph-mon[33536]: from='client.56903 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:04.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:04 smithi178 ceph-mon[39811]: from='client.56903 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:04 smithi178 ceph-mon[39811]: osdmap e328: 8 total, 8 up, 8 in 2024-02-14T17:49:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:04 smithi178 ceph-mon[39811]: pgmap v450: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:49:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:04 smithi178 ceph-mon[30321]: from='client.56903 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:04 smithi178 ceph-mon[30321]: osdmap e328: 8 total, 8 up, 8 in 2024-02-14T17:49:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:04 smithi178 ceph-mon[30321]: pgmap v450: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:49:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:04 smithi184 ceph-mon[33536]: from='client.56903 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi178-80663-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:04 smithi184 ceph-mon[33536]: osdmap e328: 8 total, 8 up, 8 in 2024-02-14T17:49:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:04 smithi184 ceph-mon[33536]: pgmap v450: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:49:05.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:49:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:05.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[30321]: osdmap e329: 8 total, 8 up, 8 in 2024-02-14T17:49:05.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:05.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi178-80663-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45"}]: dispatch 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45"}]: dispatch 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[39811]: osdmap e329: 8 total, 8 up, 8 in 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi178-80663-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:05.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:05.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45"}]: dispatch 2024-02-14T17:49:05.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45"}]: dispatch 2024-02-14T17:49:05.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:05 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:05.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:05 smithi184 ceph-mon[33536]: osdmap e329: 8 total, 8 up, 8 in 2024-02-14T17:49:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi178-80663-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:05 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45"}]: dispatch 2024-02-14T17:49:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:05 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45"}]: dispatch 2024-02-14T17:49:05.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:05 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi178-80663-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:07 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45"}]': finished 2024-02-14T17:49:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:07 smithi178 ceph-mon[39811]: osdmap e330: 8 total, 8 up, 8 in 2024-02-14T17:49:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi178-80663-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:07 smithi178 ceph-mon[39811]: pgmap v453: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:49:07.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi178-80663-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:07.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:07 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45"}]': finished 2024-02-14T17:49:07.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:07 smithi178 ceph-mon[30321]: osdmap e330: 8 total, 8 up, 8 in 2024-02-14T17:49:07.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi178-80663-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:07.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:07 smithi178 ceph-mon[30321]: pgmap v453: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:49:07.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi178-80663-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:07.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:07 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-45"}]': finished 2024-02-14T17:49:07.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:07 smithi184 ceph-mon[33536]: osdmap e330: 8 total, 8 up, 8 in 2024-02-14T17:49:07.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi178-80663-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:07.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:07 smithi184 ceph-mon[33536]: pgmap v453: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:49:08.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:08 smithi178 ceph-mon[39811]: osdmap e331: 8 total, 8 up, 8 in 2024-02-14T17:49:08.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi178-80663-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi178-80663-61"}]': finished 2024-02-14T17:49:08.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:08 smithi178 ceph-mon[39811]: osdmap e332: 8 total, 8 up, 8 in 2024-02-14T17:49:08.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:08.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:08 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:08.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:08 smithi178 ceph-mon[30321]: osdmap e331: 8 total, 8 up, 8 in 2024-02-14T17:49:08.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi178-80663-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi178-80663-61"}]': finished 2024-02-14T17:49:08.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:08 smithi178 ceph-mon[30321]: osdmap e332: 8 total, 8 up, 8 in 2024-02-14T17:49:08.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:08.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:08 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:08.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:08 smithi184 ceph-mon[33536]: osdmap e331: 8 total, 8 up, 8 in 2024-02-14T17:49:08.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi178-80663-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi178-80663-61"}]': finished 2024-02-14T17:49:08.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:08 smithi184 ceph-mon[33536]: osdmap e332: 8 total, 8 up, 8 in 2024-02-14T17:49:08.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:08.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:08 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:09 smithi178 ceph-mon[39811]: pgmap v456: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:49:09.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:09 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:09.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:09 smithi178 ceph-mon[39811]: osdmap e333: 8 total, 8 up, 8 in 2024-02-14T17:49:09.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:09.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:09 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:09.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:09 smithi178 ceph-mon[30321]: pgmap v456: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:49:09.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:09 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:09.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:09 smithi178 ceph-mon[30321]: osdmap e333: 8 total, 8 up, 8 in 2024-02-14T17:49:09.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:09.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:09 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:09.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:09 smithi184 ceph-mon[33536]: pgmap v456: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:49:09.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:09 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:09.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:09 smithi184 ceph-mon[33536]: osdmap e333: 8 total, 8 up, 8 in 2024-02-14T17:49:09.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:09.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:09 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[39811]: osdmap e334: 8 total, 8 up, 8 in 2024-02-14T17:49:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[39811]: pgmap v459: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[30321]: osdmap e334: 8 total, 8 up, 8 in 2024-02-14T17:49:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[30321]: pgmap v459: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:10 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:10.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:10 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:10 smithi184 ceph-mon[33536]: osdmap e334: 8 total, 8 up, 8 in 2024-02-14T17:49:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:10 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:10 smithi184 ceph-mon[33536]: pgmap v459: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:10 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:11.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:49:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:49:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi178-80663-61"}]': finished 2024-02-14T17:49:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-47"}]': finished 2024-02-14T17:49:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[39811]: osdmap e335: 8 total, 8 up, 8 in 2024-02-14T17:49:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-47", "mode": "writeback"}]: dispatch 2024-02-14T17:49:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-47", "mode": "writeback"}]: dispatch 2024-02-14T17:49:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi178-80663-61"}]': finished 2024-02-14T17:49:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-47"}]': finished 2024-02-14T17:49:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[30321]: osdmap e335: 8 total, 8 up, 8 in 2024-02-14T17:49:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-47", "mode": "writeback"}]: dispatch 2024-02-14T17:49:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-47", "mode": "writeback"}]: dispatch 2024-02-14T17:49:11.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:11 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:11.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi178-80663-61"}]': finished 2024-02-14T17:49:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:11 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-47"}]': finished 2024-02-14T17:49:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:11 smithi184 ceph-mon[33536]: osdmap e335: 8 total, 8 up, 8 in 2024-02-14T17:49:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-47", "mode": "writeback"}]: dispatch 2024-02-14T17:49:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi178-80663-61"}]: dispatch 2024-02-14T17:49:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:11 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-47", "mode": "writeback"}]: dispatch 2024-02-14T17:49:11.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:11 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi178-80663-61"}]': finished 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-47", "mode": "writeback"}]': finished 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: osdmap e336: 8 total, 8 up, 8 in 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[39811]: pgmap v462: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi178-80663-61"}]': finished 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-47", "mode": "writeback"}]': finished 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: osdmap e336: 8 total, 8 up, 8 in 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:12.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:12 smithi178 ceph-mon[30321]: pgmap v462: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:49:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1615578557' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi178-80663-61"}]': finished 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-47", "mode": "writeback"}]': finished 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: osdmap e336: 8 total, 8 up, 8 in 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:12.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:12 smithi184 ceph-mon[33536]: pgmap v462: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:49:14.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:49:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[39811]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[39811]: osdmap e337: 8 total, 8 up, 8 in 2024-02-14T17:49:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi178-80663-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[39811]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi178-80663-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:49:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[30321]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[30321]: osdmap e337: 8 total, 8 up, 8 in 2024-02-14T17:49:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi178-80663-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[30321]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi178-80663-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:14.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:13 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:49:14.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:13 smithi184 ceph-mon[33536]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:14.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:13 smithi184 ceph-mon[33536]: osdmap e337: 8 total, 8 up, 8 in 2024-02-14T17:49:14.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi178-80663-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:13 smithi184 ceph-mon[33536]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi178-80663-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:13 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:13 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:14.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:49:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[30321]: osdmap e338: 8 total, 8 up, 8 in 2024-02-14T17:49:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[30321]: pgmap v465: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:49:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[30321]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi178-80663-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi178-80663-62"}]': finished 2024-02-14T17:49:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:49:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[30321]: osdmap e339: 8 total, 8 up, 8 in 2024-02-14T17:49:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[39811]: osdmap e338: 8 total, 8 up, 8 in 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[39811]: pgmap v465: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[39811]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi178-80663-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi178-80663-62"}]': finished 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[39811]: osdmap e339: 8 total, 8 up, 8 in 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:15.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:14 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:15.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:14 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:49:15.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:14 smithi184 ceph-mon[33536]: osdmap e338: 8 total, 8 up, 8 in 2024-02-14T17:49:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:14 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:14 smithi184 ceph-mon[33536]: pgmap v465: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:49:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:14 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:14 smithi184 ceph-mon[33536]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi178-80663-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi178-80663-62"}]': finished 2024-02-14T17:49:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:14 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:49:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:14 smithi184 ceph-mon[33536]: osdmap e339: 8 total, 8 up, 8 in 2024-02-14T17:49:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:15.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:14 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:15.984 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:49:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:49:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:15.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:15 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:15.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:49:15.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:15 smithi178 ceph-mon[30321]: osdmap e340: 8 total, 8 up, 8 in 2024-02-14T17:49:15.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:49:15.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:49:15.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:15 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:15.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:49:15.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:15 smithi178 ceph-mon[39811]: osdmap e340: 8 total, 8 up, 8 in 2024-02-14T17:49:15.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:49:15.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:49:16.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:15 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:16.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:49:16.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:15 smithi184 ceph-mon[33536]: osdmap e340: 8 total, 8 up, 8 in 2024-02-14T17:49:16.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:49:16.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:49:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[30321]: pgmap v468: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T17:49:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[30321]: osdmap e341: 8 total, 8 up, 8 in 2024-02-14T17:49:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:49:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:49:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[30321]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[39811]: pgmap v468: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T17:49:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[39811]: osdmap e341: 8 total, 8 up, 8 in 2024-02-14T17:49:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:49:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:49:17.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:16 smithi178 ceph-mon[39811]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:17.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:16 smithi184 ceph-mon[33536]: pgmap v468: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:17.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T17:49:17.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:16 smithi184 ceph-mon[33536]: osdmap e341: 8 total, 8 up, 8 in 2024-02-14T17:49:17.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:49:17.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:17.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:49:17.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:16 smithi184 ceph-mon[33536]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T17:49:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[39811]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]': finished 2024-02-14T17:49:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[39811]: osdmap e342: 8 total, 8 up, 8 in 2024-02-14T17:49:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[39811]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[39811]: pgmap v471: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T17:49:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[30321]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]': finished 2024-02-14T17:49:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[30321]: osdmap e342: 8 total, 8 up, 8 in 2024-02-14T17:49:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[30321]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:18.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:18 smithi178 ceph-mon[30321]: pgmap v471: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:18 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T17:49:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:18 smithi184 ceph-mon[33536]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi178-80663-62"}]': finished 2024-02-14T17:49:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:18 smithi184 ceph-mon[33536]: osdmap e342: 8 total, 8 up, 8 in 2024-02-14T17:49:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3442725503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:18 smithi184 ceph-mon[33536]: from='client.56915 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]: dispatch 2024-02-14T17:49:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:18 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:18.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:18 smithi184 ceph-mon[33536]: pgmap v471: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[39811]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]': finished 2024-02-14T17:49:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:49:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[39811]: osdmap e343: 8 total, 8 up, 8 in 2024-02-14T17:49:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[39811]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[39811]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi178-80663-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[39811]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi178-80663-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[30321]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]': finished 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[30321]: osdmap e343: 8 total, 8 up, 8 in 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[30321]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[30321]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi178-80663-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:19.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:19 smithi178 ceph-mon[30321]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi178-80663-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:19.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:19 smithi184 ceph-mon[33536]: from='client.56915 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi178-80663-62"}]': finished 2024-02-14T17:49:19.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:49:19.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:19 smithi184 ceph-mon[33536]: osdmap e343: 8 total, 8 up, 8 in 2024-02-14T17:49:19.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:19.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:19.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:19 smithi184 ceph-mon[33536]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:19 smithi184 ceph-mon[33536]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:19.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi178-80663-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:19.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:19 smithi184 ceph-mon[33536]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi178-80663-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:21.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]': finished 2024-02-14T17:49:21.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[39811]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi178-80663-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:21.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[39811]: osdmap e344: 8 total, 8 up, 8 in 2024-02-14T17:49:21.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi178-80663-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:21.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[39811]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi178-80663-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:21.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:21.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:21.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:21.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:21.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[39811]: pgmap v474: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:21.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]': finished 2024-02-14T17:49:21.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[30321]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi178-80663-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:21.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[30321]: osdmap e344: 8 total, 8 up, 8 in 2024-02-14T17:49:21.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi178-80663-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:21.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[30321]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi178-80663-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:21.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:21.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:21.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:21.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:21.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:21 smithi178 ceph-mon[30321]: pgmap v474: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:21.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:49:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:49:21.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:21 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]': finished 2024-02-14T17:49:21.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:21 smithi184 ceph-mon[33536]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi178-80663-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:21.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:21 smithi184 ceph-mon[33536]: osdmap e344: 8 total, 8 up, 8 in 2024-02-14T17:49:21.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi178-80663-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:21.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:21 smithi184 ceph-mon[33536]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi178-80663-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:21.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:21.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:21 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:21.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:21.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:21 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-47"}]: dispatch 2024-02-14T17:49:21.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:21 smithi184 ceph-mon[33536]: pgmap v474: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[39811]: osdmap e345: 8 total, 8 up, 8 in 2024-02-14T17:49:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[39811]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi178-80663-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi178-80663-63"}]': finished 2024-02-14T17:49:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[39811]: osdmap e346: 8 total, 8 up, 8 in 2024-02-14T17:49:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[30321]: osdmap e345: 8 total, 8 up, 8 in 2024-02-14T17:49:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:22.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[30321]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi178-80663-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi178-80663-63"}]': finished 2024-02-14T17:49:22.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[30321]: osdmap e346: 8 total, 8 up, 8 in 2024-02-14T17:49:22.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:22 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:22 smithi184 ceph-mon[33536]: osdmap e345: 8 total, 8 up, 8 in 2024-02-14T17:49:22.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:22 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:22.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:22 smithi184 ceph-mon[33536]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi178-80663-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi178-80663-63"}]': finished 2024-02-14T17:49:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:22 smithi184 ceph-mon[33536]: osdmap e346: 8 total, 8 up, 8 in 2024-02-14T17:49:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:22 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:23 smithi178 ceph-mon[39811]: pgmap v477: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:49:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:23 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:23 smithi178 ceph-mon[39811]: osdmap e347: 8 total, 8 up, 8 in 2024-02-14T17:49:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:23 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:23 smithi178 ceph-mon[30321]: pgmap v477: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:49:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:23 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:23 smithi178 ceph-mon[30321]: osdmap e347: 8 total, 8 up, 8 in 2024-02-14T17:49:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:23 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:23 smithi184 ceph-mon[33536]: pgmap v477: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:49:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:23 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:23.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:23 smithi184 ceph-mon[33536]: osdmap e347: 8 total, 8 up, 8 in 2024-02-14T17:49:23.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:23.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:23 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:24.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:24.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:24.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[39811]: osdmap e348: 8 total, 8 up, 8 in 2024-02-14T17:49:24.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:24.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:24.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:24.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[39811]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:24.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:24.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:24.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[30321]: osdmap e348: 8 total, 8 up, 8 in 2024-02-14T17:49:24.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:24.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:24.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:24.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:24 smithi178 ceph-mon[30321]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:24.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:24 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:24.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:24 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:24.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:24 smithi184 ceph-mon[33536]: osdmap e348: 8 total, 8 up, 8 in 2024-02-14T17:49:24.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:24.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:24 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:24.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:24.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:24 smithi184 ceph-mon[33536]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[39811]: pgmap v480: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:49:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-49"}]': finished 2024-02-14T17:49:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[39811]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]': finished 2024-02-14T17:49:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[39811]: osdmap e349: 8 total, 8 up, 8 in 2024-02-14T17:49:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:49:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:49:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[39811]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:25.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[30321]: pgmap v480: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:49:25.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-49"}]': finished 2024-02-14T17:49:25.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[30321]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]': finished 2024-02-14T17:49:25.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[30321]: osdmap e349: 8 total, 8 up, 8 in 2024-02-14T17:49:25.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:49:25.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:25.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:49:25.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-mon[30321]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:25.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:25 smithi184 ceph-mon[33536]: pgmap v480: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:49:25.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-49"}]': finished 2024-02-14T17:49:25.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:25 smithi184 ceph-mon[33536]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi178-80663-63"}]': finished 2024-02-14T17:49:25.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:25 smithi184 ceph-mon[33536]: osdmap e349: 8 total, 8 up, 8 in 2024-02-14T17:49:25.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:49:25.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3893048125' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:25.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:25 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:49:25.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:25 smithi184 ceph-mon[33536]: from='client.56921 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]: dispatch 2024-02-14T17:49:26.075 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:49:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:49:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:26.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:26.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-49", "mode": "readproxy"}]': finished 2024-02-14T17:49:26.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[39811]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]': finished 2024-02-14T17:49:26.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[39811]: osdmap e350: 8 total, 8 up, 8 in 2024-02-14T17:49:26.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:26.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:26.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi178-80663-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:26.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:26.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-49", "mode": "readproxy"}]': finished 2024-02-14T17:49:26.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[30321]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]': finished 2024-02-14T17:49:26.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[30321]: osdmap e350: 8 total, 8 up, 8 in 2024-02-14T17:49:26.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:26.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:26.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi178-80663-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:26.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:26 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:26.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:26 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-49", "mode": "readproxy"}]': finished 2024-02-14T17:49:26.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:26 smithi184 ceph-mon[33536]: from='client.56921 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi178-80663-63"}]': finished 2024-02-14T17:49:26.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:26 smithi184 ceph-mon[33536]: osdmap e350: 8 total, 8 up, 8 in 2024-02-14T17:49:26.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:26.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:26.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi178-80663-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:27.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:27 smithi178 ceph-mon[39811]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:27.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi178-80663-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:27.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:27 smithi178 ceph-mon[39811]: osdmap e351: 8 total, 8 up, 8 in 2024-02-14T17:49:27.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi178-80663-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:27 smithi178 ceph-mon[30321]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi178-80663-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:27 smithi178 ceph-mon[30321]: osdmap e351: 8 total, 8 up, 8 in 2024-02-14T17:49:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi178-80663-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:27 smithi184 ceph-mon[33536]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:27.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi178-80663-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:27.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:27 smithi184 ceph-mon[33536]: osdmap e351: 8 total, 8 up, 8 in 2024-02-14T17:49:27.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi178-80663-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:28 smithi184 ceph-mon[33536]: osdmap e352: 8 total, 8 up, 8 in 2024-02-14T17:49:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:28 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:28 smithi184 ceph-mon[33536]: pgmap v486: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:29.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:28 smithi178 ceph-mon[39811]: osdmap e352: 8 total, 8 up, 8 in 2024-02-14T17:49:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:28 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:28 smithi178 ceph-mon[39811]: pgmap v486: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:28 smithi178 ceph-mon[30321]: osdmap e352: 8 total, 8 up, 8 in 2024-02-14T17:49:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:28 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:28 smithi178 ceph-mon[30321]: pgmap v486: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:29.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi178-80663-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi178-80663-64"}]': finished 2024-02-14T17:49:29.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:29 smithi184 ceph-mon[33536]: osdmap e353: 8 total, 8 up, 8 in 2024-02-14T17:49:30.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi178-80663-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi178-80663-64"}]': finished 2024-02-14T17:49:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:29 smithi178 ceph-mon[39811]: osdmap e353: 8 total, 8 up, 8 in 2024-02-14T17:49:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi178-80663-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi178-80663-64"}]': finished 2024-02-14T17:49:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:29 smithi178 ceph-mon[30321]: osdmap e353: 8 total, 8 up, 8 in 2024-02-14T17:49:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:30 smithi184 ceph-mon[33536]: osdmap e354: 8 total, 8 up, 8 in 2024-02-14T17:49:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:30 smithi184 ceph-mon[33536]: pgmap v489: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:30.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:30 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:31.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:30 smithi178 ceph-mon[39811]: osdmap e354: 8 total, 8 up, 8 in 2024-02-14T17:49:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:30 smithi178 ceph-mon[39811]: pgmap v489: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:30 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:30 smithi178 ceph-mon[30321]: osdmap e354: 8 total, 8 up, 8 in 2024-02-14T17:49:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:30 smithi178 ceph-mon[30321]: pgmap v489: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:30 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:31.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:49:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:49:31.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:31 smithi184 ceph-mon[33536]: osdmap e355: 8 total, 8 up, 8 in 2024-02-14T17:49:31.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:31 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:31 smithi178 ceph-mon[39811]: osdmap e355: 8 total, 8 up, 8 in 2024-02-14T17:49:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:31 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:31 smithi178 ceph-mon[30321]: osdmap e355: 8 total, 8 up, 8 in 2024-02-14T17:49:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:31 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi178-80663-64"}]': finished 2024-02-14T17:49:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:32 smithi184 ceph-mon[33536]: osdmap e356: 8 total, 8 up, 8 in 2024-02-14T17:49:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:32.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:32 smithi184 ceph-mon[33536]: pgmap v492: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:49:32.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi178-80663-64"}]': finished 2024-02-14T17:49:33.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi178-80663-64"}]': finished 2024-02-14T17:49:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:32 smithi178 ceph-mon[30321]: osdmap e356: 8 total, 8 up, 8 in 2024-02-14T17:49:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:32 smithi178 ceph-mon[30321]: pgmap v492: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:49:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi178-80663-64"}]': finished 2024-02-14T17:49:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi178-80663-64"}]': finished 2024-02-14T17:49:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:32 smithi178 ceph-mon[39811]: osdmap e356: 8 total, 8 up, 8 in 2024-02-14T17:49:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi178-80663-64"}]: dispatch 2024-02-14T17:49:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:32 smithi178 ceph-mon[39811]: pgmap v492: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:49:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1077413972' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi178-80663-64"}]': finished 2024-02-14T17:49:33.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:33 smithi184 ceph-mon[33536]: osdmap e357: 8 total, 8 up, 8 in 2024-02-14T17:49:33.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:33.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:33 smithi184 ceph-mon[33536]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:33.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:33.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:33 smithi184 ceph-mon[33536]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:33.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi178-80663-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:33.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:33 smithi184 ceph-mon[33536]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi178-80663-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:33.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:33 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:49:33.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:33 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:49:33.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:33 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:49:33.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:33 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:34.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[39811]: osdmap e357: 8 total, 8 up, 8 in 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[39811]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[39811]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi178-80663-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[39811]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi178-80663-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[30321]: osdmap e357: 8 total, 8 up, 8 in 2024-02-14T17:49:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[30321]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[30321]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi178-80663-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[30321]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi178-80663-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:49:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:49:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:49:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:33 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:34.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:34 smithi184 ceph-mon[33536]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi178-80663-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:34.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:34 smithi184 ceph-mon[33536]: osdmap e358: 8 total, 8 up, 8 in 2024-02-14T17:49:34.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi178-80663-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:34.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:34 smithi184 ceph-mon[33536]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi178-80663-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:34.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:34 smithi184 ceph-mon[33536]: pgmap v495: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:49:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:34 smithi178 ceph-mon[39811]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi178-80663-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:34 smithi178 ceph-mon[39811]: osdmap e358: 8 total, 8 up, 8 in 2024-02-14T17:49:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi178-80663-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:34 smithi178 ceph-mon[39811]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi178-80663-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:34 smithi178 ceph-mon[39811]: pgmap v495: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:49:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:34 smithi178 ceph-mon[30321]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi178-80663-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:34 smithi178 ceph-mon[30321]: osdmap e358: 8 total, 8 up, 8 in 2024-02-14T17:49:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi178-80663-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:34 smithi178 ceph-mon[30321]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi178-80663-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:34 smithi178 ceph-mon[30321]: pgmap v495: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:49:35.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:35 smithi184 ceph-mon[33536]: osdmap e359: 8 total, 8 up, 8 in 2024-02-14T17:49:36.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:35 smithi178 ceph-mon[39811]: osdmap e359: 8 total, 8 up, 8 in 2024-02-14T17:49:36.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:35 smithi178 ceph-mon[30321]: osdmap e359: 8 total, 8 up, 8 in 2024-02-14T17:49:36.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:49:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:49:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:36.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:36 smithi184 ceph-mon[33536]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi178-80663-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi178-80663-65"}]': finished 2024-02-14T17:49:36.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:36 smithi184 ceph-mon[33536]: osdmap e360: 8 total, 8 up, 8 in 2024-02-14T17:49:36.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:36.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:36.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:36 smithi184 ceph-mon[33536]: pgmap v498: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:36 smithi178 ceph-mon[39811]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi178-80663-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi178-80663-65"}]': finished 2024-02-14T17:49:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:36 smithi178 ceph-mon[39811]: osdmap e360: 8 total, 8 up, 8 in 2024-02-14T17:49:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:37.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:36 smithi178 ceph-mon[39811]: pgmap v498: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:37.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:36 smithi178 ceph-mon[30321]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi178-80663-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi178-80663-65"}]': finished 2024-02-14T17:49:37.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:36 smithi178 ceph-mon[30321]: osdmap e360: 8 total, 8 up, 8 in 2024-02-14T17:49:37.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:37.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:37.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:36 smithi178 ceph-mon[30321]: pgmap v498: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:37.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:37 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:37.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:37 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:49:37.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:37 smithi184 ceph-mon[33536]: osdmap e361: 8 total, 8 up, 8 in 2024-02-14T17:49:37.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:37.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:37 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:38.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:37 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:38.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:37 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:49:38.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:37 smithi178 ceph-mon[39811]: osdmap e361: 8 total, 8 up, 8 in 2024-02-14T17:49:38.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:38.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:37 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:38.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:37 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:38.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:37 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:49:38.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:37 smithi178 ceph-mon[30321]: osdmap e361: 8 total, 8 up, 8 in 2024-02-14T17:49:38.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:38.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:37 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:38.670 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP:163:head 2024-02-14T17:49:38.670 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:5d165639:::164:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:f43765fc:::165:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:b4c720e9:::166:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:e694b040:::167:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:afa38db2:::168:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:77ba9f53:::169:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:87495034:::170:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:7c96bf0e:::171:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:dbe346cc:::172:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:e943ec24:::173:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:f97a9c0c:::174:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:6f26e74d:::175:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:4f95e106:::176:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:0e6f2f8f:::177:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:05db05f1:::178:head 2024-02-14T17:49:38.671 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:38a78d66:::179:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:d095610b:::180:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:a1a9d709:::181:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:1e5d39db:::182:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:f7df4fb9:::183:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:03a7f161:::184:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:ba70721e:::185:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:28e5662d:::186:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:973d52de:::187:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:4303eb1c:::188:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:b990b48e:::189:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:29b8165b:::190:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:3547f197:::191:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:7e260936:::192:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:1abec7b1:::193:head 2024-02-14T17:49:38.672 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:10fdda93:::194:head 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:15817eea:::195:head 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:770bab57:::196:head 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:ed9e13e7:::197:head 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:71471a8f:::198:head 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: checking for 277:10fb1d02:::199:head 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8026 ms) 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,0 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: first is 1707932933 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,0 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,0 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,1707932935,1707932936,0 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,1707932935,1707932936,0 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,1707932935,1707932936,0 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,1707932935,1707932936,1707932938,1707932939,0 2024-02-14T17:49:38.673 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,1707932935,1707932936,1707932938,1707932939,0 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,1707932935,1707932936,1707932938,1707932939,0 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,1707932935,1707932936,1707932938,1707932939,1707932941,1707932942,0 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,1707932935,1707932936,1707932938,1707932939,1707932941,1707932942,0 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932933,1707932935,1707932936,1707932938,1707932939,1707932941,1707932942,0 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707932936,1707932938,1707932939,1707932941,1707932942,1707932944,1707932945,0 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: first now 1707932936, trimmed 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21295 ms) 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: foo0 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14044 ms) 2024-02-14T17:49:38.674 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-02-14T17:49:38.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:38 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:38.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:38 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]': finished 2024-02-14T17:49:38.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:38 smithi184 ceph-mon[33536]: osdmap e362: 8 total, 8 up, 8 in 2024-02-14T17:49:38.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:38.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:38 smithi184 ceph-mon[33536]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:38.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:38.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:38 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:38.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:38.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:38 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:38.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:38 smithi184 ceph-mon[33536]: pgmap v501: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]': finished 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[30321]: osdmap e362: 8 total, 8 up, 8 in 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[30321]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[30321]: pgmap v501: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:39.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]': finished 2024-02-14T17:49:39.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[39811]: osdmap e362: 8 total, 8 up, 8 in 2024-02-14T17:49:39.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:39.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[39811]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:39.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:39.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:39.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:39.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-49"}]: dispatch 2024-02-14T17:49:39.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:38 smithi178 ceph-mon[39811]: pgmap v501: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:39.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:39 smithi184 ceph-mon[33536]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]': finished 2024-02-14T17:49:39.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:39 smithi184 ceph-mon[33536]: osdmap e363: 8 total, 8 up, 8 in 2024-02-14T17:49:39.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:39.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:39 smithi184 ceph-mon[33536]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:40.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:39 smithi178 ceph-mon[39811]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]': finished 2024-02-14T17:49:40.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:39 smithi178 ceph-mon[39811]: osdmap e363: 8 total, 8 up, 8 in 2024-02-14T17:49:40.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:40.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:39 smithi178 ceph-mon[39811]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:40.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:39 smithi178 ceph-mon[30321]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi178-80663-65"}]': finished 2024-02-14T17:49:40.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:39 smithi178 ceph-mon[30321]: osdmap e363: 8 total, 8 up, 8 in 2024-02-14T17:49:40.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1250280684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:40.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:39 smithi178 ceph-mon[30321]: from='client.49384 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]: dispatch 2024-02-14T17:49:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:40 smithi184 ceph-mon[33536]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]': finished 2024-02-14T17:49:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:40 smithi184 ceph-mon[33536]: osdmap e364: 8 total, 8 up, 8 in 2024-02-14T17:49:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:40 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:40 smithi184 ceph-mon[33536]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:40 smithi184 ceph-mon[33536]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi178-80663-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:40 smithi184 ceph-mon[33536]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi178-80663-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:40.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:40 smithi184 ceph-mon[33536]: pgmap v504: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[39811]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]': finished 2024-02-14T17:49:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[39811]: osdmap e364: 8 total, 8 up, 8 in 2024-02-14T17:49:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[39811]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[39811]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi178-80663-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[39811]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi178-80663-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:41.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[39811]: pgmap v504: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:41.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[30321]: from='client.49384 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi178-80663-65"}]': finished 2024-02-14T17:49:41.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[30321]: osdmap e364: 8 total, 8 up, 8 in 2024-02-14T17:49:41.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:41.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:41.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:41.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[30321]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:41.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:41.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[30321]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:41.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi178-80663-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:41.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[30321]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi178-80663-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:41.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:40 smithi178 ceph-mon[30321]: pgmap v504: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:41.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:49:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:49:42.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:41 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:42.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:41 smithi184 ceph-mon[33536]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi178-80663-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:42.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:41 smithi184 ceph-mon[33536]: osdmap e365: 8 total, 8 up, 8 in 2024-02-14T17:49:42.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi178-80663-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:42.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:41 smithi184 ceph-mon[33536]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi178-80663-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:42.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:42.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:41 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:42.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:41 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:42.221 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:41 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:42.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:42.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[30321]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi178-80663-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:42.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[30321]: osdmap e365: 8 total, 8 up, 8 in 2024-02-14T17:49:42.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi178-80663-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:42.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[30321]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi178-80663-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:42.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:42.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:42.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:42.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:42.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:42.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[39811]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi178-80663-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:42.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[39811]: osdmap e365: 8 total, 8 up, 8 in 2024-02-14T17:49:42.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi178-80663-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:42.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[39811]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi178-80663-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:42.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:42.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:42.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:42.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:41 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:49:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:42 smithi184 ceph-mon[33536]: osdmap e366: 8 total, 8 up, 8 in 2024-02-14T17:49:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:42 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:49:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:42 smithi184 ceph-mon[33536]: pgmap v507: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:42 smithi184 ceph-mon[33536]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi178-80663-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi178-80663-66"}]': finished 2024-02-14T17:49:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:42 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-51"}]': finished 2024-02-14T17:49:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:42 smithi184 ceph-mon[33536]: osdmap e367: 8 total, 8 up, 8 in 2024-02-14T17:49:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-51", "mode": "writeback"}]: dispatch 2024-02-14T17:49:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:42 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-51", "mode": "writeback"}]: dispatch 2024-02-14T17:49:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:42 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:43.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:42 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:49:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[39811]: osdmap e366: 8 total, 8 up, 8 in 2024-02-14T17:49:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:49:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[39811]: pgmap v507: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[39811]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi178-80663-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi178-80663-66"}]': finished 2024-02-14T17:49:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-51"}]': finished 2024-02-14T17:49:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[39811]: osdmap e367: 8 total, 8 up, 8 in 2024-02-14T17:49:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-51", "mode": "writeback"}]: dispatch 2024-02-14T17:49:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-51", "mode": "writeback"}]: dispatch 2024-02-14T17:49:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:43.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:43.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:49:43.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[30321]: osdmap e366: 8 total, 8 up, 8 in 2024-02-14T17:49:43.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:49:43.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[30321]: pgmap v507: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:43.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[30321]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi178-80663-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi178-80663-66"}]': finished 2024-02-14T17:49:43.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-51"}]': finished 2024-02-14T17:49:43.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[30321]: osdmap e367: 8 total, 8 up, 8 in 2024-02-14T17:49:43.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-51", "mode": "writeback"}]: dispatch 2024-02-14T17:49:43.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-51", "mode": "writeback"}]: dispatch 2024-02-14T17:49:43.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:43.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:42 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:44.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:43 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:43 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-51", "mode": "writeback"}]': finished 2024-02-14T17:49:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:43 smithi184 ceph-mon[33536]: osdmap e368: 8 total, 8 up, 8 in 2024-02-14T17:49:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:44.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:44.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:44.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:44.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-51", "mode": "writeback"}]': finished 2024-02-14T17:49:44.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[39811]: osdmap e368: 8 total, 8 up, 8 in 2024-02-14T17:49:44.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:44.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-51", "mode": "writeback"}]': finished 2024-02-14T17:49:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[30321]: osdmap e368: 8 total, 8 up, 8 in 2024-02-14T17:49:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:49:45.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:44 smithi184 ceph-mon[33536]: pgmap v510: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:44 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:49:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:44 smithi184 ceph-mon[33536]: osdmap e369: 8 total, 8 up, 8 in 2024-02-14T17:49:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:44 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:45.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:44 smithi184 ceph-mon[33536]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:45.234 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[30321]: pgmap v510: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:45.234 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:49:45.234 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[30321]: osdmap e369: 8 total, 8 up, 8 in 2024-02-14T17:49:45.234 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:45.234 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:45.234 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:45.234 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[30321]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:45.235 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[39811]: pgmap v510: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:45.235 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:49:45.235 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[39811]: osdmap e369: 8 total, 8 up, 8 in 2024-02-14T17:49:45.235 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:45.235 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:49:45.235 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:45.235 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:44 smithi178 ceph-mon[39811]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:46.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:49:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:49:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:46.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ 2 2024-02-14T17:49:46.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3013 ms) 2024-02-14T17:49:46.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-14T17:49:46.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7016 ms) 2024-02-14T17:49:46.700 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7026 ms) 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7013 ms) 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7029 ms) 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7061 ms) 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7016 ms) 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139286 ms total) 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (302650 ms total) 2024-02-14T17:49:46.701 INFO:tasks.workunit.client.0.smithi178.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-02-14T17:49:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:46 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:49:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:46 smithi184 ceph-mon[33536]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]': finished 2024-02-14T17:49:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:46 smithi184 ceph-mon[33536]: osdmap e370: 8 total, 8 up, 8 in 2024-02-14T17:49:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:46.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:46 smithi184 ceph-mon[33536]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:46.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:46 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:46.971 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:46 smithi184 ceph-mon[33536]: pgmap v513: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-14T17:49:47.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[39811]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]': finished 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[39811]: osdmap e370: 8 total, 8 up, 8 in 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[39811]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[39811]: pgmap v513: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[30321]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi178-80663-66"}]': finished 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[30321]: osdmap e370: 8 total, 8 up, 8 in 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2748211224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[30321]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]: dispatch 2024-02-14T17:49:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:49:47.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:46 smithi178 ceph-mon[30321]: pgmap v513: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-14T17:49:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:47 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:47 smithi184 ceph-mon[33536]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]': finished 2024-02-14T17:49:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:47 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:49:47.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:47 smithi184 ceph-mon[33536]: osdmap e371: 8 total, 8 up, 8 in 2024-02-14T17:49:47.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:47.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:47 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:48.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[39811]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]': finished 2024-02-14T17:49:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:49:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[39811]: osdmap e371: 8 total, 8 up, 8 in 2024-02-14T17:49:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[30321]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi178-80663-66"}]': finished 2024-02-14T17:49:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:49:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[30321]: osdmap e371: 8 total, 8 up, 8 in 2024-02-14T17:49:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:47 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:49:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:49:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:48 smithi184 ceph-mon[33536]: osdmap e372: 8 total, 8 up, 8 in 2024-02-14T17:49:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:49:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:49:48.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:48 smithi184 ceph-mon[33536]: pgmap v516: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-14T17:49:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:49:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:48 smithi178 ceph-mon[39811]: osdmap e372: 8 total, 8 up, 8 in 2024-02-14T17:49:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:49:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:49:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:48 smithi178 ceph-mon[39811]: pgmap v516: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-14T17:49:49.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:49:49.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:48 smithi178 ceph-mon[30321]: osdmap e372: 8 total, 8 up, 8 in 2024-02-14T17:49:49.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:49:49.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:49:49.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:48 smithi178 ceph-mon[30321]: pgmap v516: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-14T17:49:49.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:49 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "target_max_objects","val": "1"}]': finished 2024-02-14T17:49:49.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:49 smithi184 ceph-mon[33536]: osdmap e373: 8 total, 8 up, 8 in 2024-02-14T17:49:50.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:49 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "target_max_objects","val": "1"}]': finished 2024-02-14T17:49:50.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:49 smithi178 ceph-mon[39811]: osdmap e373: 8 total, 8 up, 8 in 2024-02-14T17:49:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:49 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-51","var": "target_max_objects","val": "1"}]': finished 2024-02-14T17:49:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:49 smithi178 ceph-mon[30321]: osdmap e373: 8 total, 8 up, 8 in 2024-02-14T17:49:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:50 smithi184 ceph-mon[33536]: pgmap v518: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:51.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:50 smithi178 ceph-mon[39811]: pgmap v518: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:50 smithi178 ceph-mon[30321]: pgmap v518: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:51.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:49:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:49:51.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:51 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T17:49:51.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:51 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:51 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T17:49:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:51 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:51 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T17:49:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:51 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:52 smithi184 ceph-mon[33536]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:49:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:52 smithi178 ceph-mon[39811]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:49:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:52 smithi178 ceph-mon[30321]: pgmap v519: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:49:55.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:55 smithi178 ceph-mon[30321]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 678 B/s rd, 0 B/s wr, 0 op/s 2024-02-14T17:49:55.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:55 smithi178 ceph-mon[39811]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 678 B/s rd, 0 B/s wr, 0 op/s 2024-02-14T17:49:55.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:55 smithi184 ceph-mon[33536]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 678 B/s rd, 0 B/s wr, 0 op/s 2024-02-14T17:49:56.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:49:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:49:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:57.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:57 smithi178 ceph-mon[39811]: pgmap v521: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:49:57.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:57 smithi178 ceph-mon[30321]: pgmap v521: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:49:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:57 smithi184 ceph-mon[33536]: pgmap v521: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:49:59.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:58 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:58 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:58 smithi184 ceph-mon[33536]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:49:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:59.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:58 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:58 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:58 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:58 smithi178 ceph-mon[39811]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:49:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:58 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:58 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:49:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:58 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:58 smithi178 ceph-mon[30321]: pgmap v522: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:49:59.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:49:59.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:58 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:00.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:59 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:50:00.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:59 smithi184 ceph-mon[33536]: osdmap e374: 8 total, 8 up, 8 in 2024-02-14T17:50:00.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:00.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:49:59 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:00.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:59 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:50:00.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:59 smithi178 ceph-mon[30321]: osdmap e374: 8 total, 8 up, 8 in 2024-02-14T17:50:00.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:00.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:49:59 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:59 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:50:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:59 smithi178 ceph-mon[39811]: osdmap e374: 8 total, 8 up, 8 in 2024-02-14T17:50:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:00.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:49:59 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]': finished 2024-02-14T17:50:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: osdmap e375: 8 total, 8 up, 8 in 2024-02-14T17:50:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: Health detail: HEALTH_WARN 1 cache pools at or near target size; 1 pool(s) do not have an application enabled 2024-02-14T17:50:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: [WRN] CACHE_POOL_NEAR_FULL: 1 cache pools at or near target size 2024-02-14T17:50:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: cache pool 'test-rados-api-smithi178-80903-51' with 2 objects at/near target max 1 objects 2024-02-14T17:50:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-14T17:50:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T17:50:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T17:50:01.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:00 smithi184 ceph-mon[33536]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]': finished 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: osdmap e375: 8 total, 8 up, 8 in 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: Health detail: HEALTH_WARN 1 cache pools at or near target size; 1 pool(s) do not have an application enabled 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: [WRN] CACHE_POOL_NEAR_FULL: 1 cache pools at or near target size 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: cache pool 'test-rados-api-smithi178-80903-51' with 2 objects at/near target max 1 objects 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T17:50:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[39811]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]': finished 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: osdmap e375: 8 total, 8 up, 8 in 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-51"}]: dispatch 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: Health detail: HEALTH_WARN 1 cache pools at or near target size; 1 pool(s) do not have an application enabled 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: [WRN] CACHE_POOL_NEAR_FULL: 1 cache pools at or near target size 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: cache pool 'test-rados-api-smithi178-80903-51' with 2 objects at/near target max 1 objects 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T17:50:01.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:00 smithi178 ceph-mon[30321]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T17:50:01.470 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:50:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:50:02.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:01 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T17:50:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:01 smithi184 ceph-mon[33536]: osdmap e376: 8 total, 8 up, 8 in 2024-02-14T17:50:02.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:01 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:01 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T17:50:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:01 smithi178 ceph-mon[39811]: osdmap e376: 8 total, 8 up, 8 in 2024-02-14T17:50:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:01 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:02.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:01 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T17:50:02.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:01 smithi178 ceph-mon[30321]: osdmap e376: 8 total, 8 up, 8 in 2024-02-14T17:50:02.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:01 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:02 smithi184 ceph-mon[33536]: osdmap e377: 8 total, 8 up, 8 in 2024-02-14T17:50:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:02 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:03.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:02 smithi184 ceph-mon[33536]: pgmap v528: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:50:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:02 smithi178 ceph-mon[39811]: osdmap e377: 8 total, 8 up, 8 in 2024-02-14T17:50:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:02 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:03.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:02 smithi178 ceph-mon[39811]: pgmap v528: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:50:03.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:02 smithi178 ceph-mon[30321]: osdmap e377: 8 total, 8 up, 8 in 2024-02-14T17:50:03.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:03.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:02 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:03.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:02 smithi178 ceph-mon[30321]: pgmap v528: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:50:04.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:03 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:04.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:03 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:04.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:03 smithi184 ceph-mon[33536]: osdmap e378: 8 total, 8 up, 8 in 2024-02-14T17:50:04.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:04.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:03 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:04.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-53"}]: dispatch 2024-02-14T17:50:04.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:03 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-53"}]: dispatch 2024-02-14T17:50:04.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:04.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:04.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[39811]: osdmap e378: 8 total, 8 up, 8 in 2024-02-14T17:50:04.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:04.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:04.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-53"}]: dispatch 2024-02-14T17:50:04.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-53"}]: dispatch 2024-02-14T17:50:04.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:04.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:04.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[30321]: osdmap e378: 8 total, 8 up, 8 in 2024-02-14T17:50:04.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:04.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:04.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-53"}]: dispatch 2024-02-14T17:50:04.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:03 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-53"}]: dispatch 2024-02-14T17:50:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:04 smithi184 ceph-mon[33536]: osdmap e379: 8 total, 8 up, 8 in 2024-02-14T17:50:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:04 smithi184 ceph-mon[33536]: pgmap v531: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:50:05.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:04 smithi178 ceph-mon[30321]: osdmap e379: 8 total, 8 up, 8 in 2024-02-14T17:50:05.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:04 smithi178 ceph-mon[30321]: pgmap v531: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:50:05.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:04 smithi178 ceph-mon[39811]: osdmap e379: 8 total, 8 up, 8 in 2024-02-14T17:50:05.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:04 smithi178 ceph-mon[39811]: pgmap v531: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:50:05.955 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:50:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:50:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:06.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:05 smithi184 ceph-mon[33536]: osdmap e380: 8 total, 8 up, 8 in 2024-02-14T17:50:06.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:06.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:05 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:06.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:05 smithi178 ceph-mon[39811]: osdmap e380: 8 total, 8 up, 8 in 2024-02-14T17:50:06.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:06.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:05 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:05 smithi178 ceph-mon[30321]: osdmap e380: 8 total, 8 up, 8 in 2024-02-14T17:50:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:06.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:05 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:07.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:07.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:06 smithi184 ceph-mon[33536]: osdmap e381: 8 total, 8 up, 8 in 2024-02-14T17:50:07.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:07.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:07.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:07.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-55"}]: dispatch 2024-02-14T17:50:07.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-55"}]: dispatch 2024-02-14T17:50:07.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:06 smithi184 ceph-mon[33536]: pgmap v534: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:50:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[39811]: osdmap e381: 8 total, 8 up, 8 in 2024-02-14T17:50:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-55"}]: dispatch 2024-02-14T17:50:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-55"}]: dispatch 2024-02-14T17:50:07.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[39811]: pgmap v534: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:50:07.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:07.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[30321]: osdmap e381: 8 total, 8 up, 8 in 2024-02-14T17:50:07.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:07.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:07.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:07.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-55"}]: dispatch 2024-02-14T17:50:07.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-55"}]: dispatch 2024-02-14T17:50:07.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:06 smithi178 ceph-mon[30321]: pgmap v534: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:50:08.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:07 smithi184 ceph-mon[33536]: osdmap e382: 8 total, 8 up, 8 in 2024-02-14T17:50:08.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:07 smithi178 ceph-mon[39811]: osdmap e382: 8 total, 8 up, 8 in 2024-02-14T17:50:08.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:07 smithi178 ceph-mon[30321]: osdmap e382: 8 total, 8 up, 8 in 2024-02-14T17:50:09.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:08 smithi178 ceph-mon[30321]: osdmap e383: 8 total, 8 up, 8 in 2024-02-14T17:50:09.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:09.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:08 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:09.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:08 smithi178 ceph-mon[30321]: pgmap v537: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:50:09.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:08 smithi178 ceph-mon[39811]: osdmap e383: 8 total, 8 up, 8 in 2024-02-14T17:50:09.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:09.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:08 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:09.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:08 smithi178 ceph-mon[39811]: pgmap v537: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:50:09.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:08 smithi184 ceph-mon[33536]: osdmap e383: 8 total, 8 up, 8 in 2024-02-14T17:50:09.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:09.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:08 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:09.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:08 smithi184 ceph-mon[33536]: pgmap v537: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:50:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[39811]: osdmap e384: 8 total, 8 up, 8 in 2024-02-14T17:50:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-57"}]: dispatch 2024-02-14T17:50:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-57"}]: dispatch 2024-02-14T17:50:10.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[30321]: osdmap e384: 8 total, 8 up, 8 in 2024-02-14T17:50:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-57"}]: dispatch 2024-02-14T17:50:10.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:09 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-57"}]: dispatch 2024-02-14T17:50:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:09 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:09 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:09 smithi184 ceph-mon[33536]: osdmap e384: 8 total, 8 up, 8 in 2024-02-14T17:50:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:09 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-57"}]: dispatch 2024-02-14T17:50:10.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:09 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-57"}]: dispatch 2024-02-14T17:50:11.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:10 smithi178 ceph-mon[39811]: osdmap e385: 8 total, 8 up, 8 in 2024-02-14T17:50:11.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:10 smithi178 ceph-mon[39811]: pgmap v540: 196 pgs: 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:11.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:10 smithi178 ceph-mon[30321]: osdmap e385: 8 total, 8 up, 8 in 2024-02-14T17:50:11.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:10 smithi178 ceph-mon[30321]: pgmap v540: 196 pgs: 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:11.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:10 smithi184 ceph-mon[33536]: osdmap e385: 8 total, 8 up, 8 in 2024-02-14T17:50:11.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:10 smithi184 ceph-mon[33536]: pgmap v540: 196 pgs: 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:11.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:50:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:50:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:11 smithi178 ceph-mon[39811]: osdmap e386: 8 total, 8 up, 8 in 2024-02-14T17:50:12.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:11 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.336 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:11 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:12.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:11 smithi178 ceph-mon[30321]: osdmap e386: 8 total, 8 up, 8 in 2024-02-14T17:50:12.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:11 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:11 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:12.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:11 smithi184 ceph-mon[33536]: osdmap e386: 8 total, 8 up, 8 in 2024-02-14T17:50:12.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:11 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:11 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[30321]: osdmap e387: 8 total, 8 up, 8 in 2024-02-14T17:50:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-59"}]: dispatch 2024-02-14T17:50:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-59"}]: dispatch 2024-02-14T17:50:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[30321]: pgmap v543: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:13.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:13.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[39811]: osdmap e387: 8 total, 8 up, 8 in 2024-02-14T17:50:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-59"}]: dispatch 2024-02-14T17:50:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-59"}]: dispatch 2024-02-14T17:50:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:13 smithi178 ceph-mon[39811]: pgmap v543: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:13 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:13 smithi184 ceph-mon[33536]: osdmap e387: 8 total, 8 up, 8 in 2024-02-14T17:50:13.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:13.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:13.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:13 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:13.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-59"}]: dispatch 2024-02-14T17:50:13.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:13 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-59"}]: dispatch 2024-02-14T17:50:13.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:13 smithi184 ceph-mon[33536]: pgmap v543: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:14 smithi178 ceph-mon[39811]: osdmap e388: 8 total, 8 up, 8 in 2024-02-14T17:50:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:14 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:50:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:14 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:14 smithi178 ceph-mon[30321]: osdmap e388: 8 total, 8 up, 8 in 2024-02-14T17:50:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:14 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:50:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:14 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:14 smithi184 ceph-mon[33536]: osdmap e388: 8 total, 8 up, 8 in 2024-02-14T17:50:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:14 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:50:14.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:14 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:15.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:15 smithi178 ceph-mon[30321]: osdmap e389: 8 total, 8 up, 8 in 2024-02-14T17:50:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:15 smithi178 ceph-mon[30321]: pgmap v546: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:15 smithi178 ceph-mon[39811]: osdmap e389: 8 total, 8 up, 8 in 2024-02-14T17:50:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:15 smithi178 ceph-mon[39811]: pgmap v546: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:15 smithi184 ceph-mon[33536]: osdmap e389: 8 total, 8 up, 8 in 2024-02-14T17:50:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:15 smithi184 ceph-mon[33536]: pgmap v546: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:16.022 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:50:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:50:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:16.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:16.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:16.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[39811]: osdmap e390: 8 total, 8 up, 8 in 2024-02-14T17:50:16.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:16.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:16.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:16.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:16.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:16.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[30321]: osdmap e390: 8 total, 8 up, 8 in 2024-02-14T17:50:16.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:16.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:16.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:16.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:16 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:16.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:16.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:16 smithi184 ceph-mon[33536]: osdmap e390: 8 total, 8 up, 8 in 2024-02-14T17:50:16.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:16.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:16.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:50:17.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[39811]: osdmap e391: 8 total, 8 up, 8 in 2024-02-14T17:50:17.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:17.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:17.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-61"}]: dispatch 2024-02-14T17:50:17.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-61"}]: dispatch 2024-02-14T17:50:17.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[39811]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:50:17.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:50:17.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[30321]: osdmap e391: 8 total, 8 up, 8 in 2024-02-14T17:50:17.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:17.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:17.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-61"}]: dispatch 2024-02-14T17:50:17.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-61"}]: dispatch 2024-02-14T17:50:17.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:17 smithi178 ceph-mon[30321]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:50:17.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:17 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:50:17.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:17 smithi184 ceph-mon[33536]: osdmap e391: 8 total, 8 up, 8 in 2024-02-14T17:50:17.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:17.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:17 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:17.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-61"}]: dispatch 2024-02-14T17:50:17.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:17 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-61"}]: dispatch 2024-02-14T17:50:17.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:17 smithi184 ceph-mon[33536]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:50:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:18 smithi178 ceph-mon[39811]: osdmap e392: 8 total, 8 up, 8 in 2024-02-14T17:50:18.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:18 smithi178 ceph-mon[30321]: osdmap e392: 8 total, 8 up, 8 in 2024-02-14T17:50:18.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:18 smithi184 ceph-mon[33536]: osdmap e392: 8 total, 8 up, 8 in 2024-02-14T17:50:19.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[39811]: osdmap e393: 8 total, 8 up, 8 in 2024-02-14T17:50:19.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:19.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:19.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[39811]: pgmap v552: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:50:19.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:19.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[39811]: osdmap e394: 8 total, 8 up, 8 in 2024-02-14T17:50:19.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[30321]: osdmap e393: 8 total, 8 up, 8 in 2024-02-14T17:50:19.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:19.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:19.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[30321]: pgmap v552: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:50:19.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:19.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:19 smithi178 ceph-mon[30321]: osdmap e394: 8 total, 8 up, 8 in 2024-02-14T17:50:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:19 smithi184 ceph-mon[33536]: osdmap e393: 8 total, 8 up, 8 in 2024-02-14T17:50:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:19.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:19 smithi184 ceph-mon[33536]: pgmap v552: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:50:19.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:19.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:19 smithi184 ceph-mon[33536]: osdmap e394: 8 total, 8 up, 8 in 2024-02-14T17:50:20.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:20.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:20.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:20 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:20.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:20 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:50:20.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:20 smithi178 ceph-mon[39811]: osdmap e395: 8 total, 8 up, 8 in 2024-02-14T17:50:20.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:20.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:20.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:20 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:20.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:20 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:50:20.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:20 smithi178 ceph-mon[30321]: osdmap e395: 8 total, 8 up, 8 in 2024-02-14T17:50:20.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:20.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:20.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:20 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:20.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:20 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:50:20.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:20 smithi184 ceph-mon[33536]: osdmap e395: 8 total, 8 up, 8 in 2024-02-14T17:50:21.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:21 smithi178 ceph-mon[39811]: pgmap v555: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:21.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:21 smithi178 ceph-mon[30321]: pgmap v555: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:21.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:21 smithi184 ceph-mon[33536]: pgmap v555: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:21.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:50:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:50:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:22 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:22 smithi178 ceph-mon[39811]: osdmap e396: 8 total, 8 up, 8 in 2024-02-14T17:50:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:22 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:22 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:22 smithi178 ceph-mon[30321]: osdmap e396: 8 total, 8 up, 8 in 2024-02-14T17:50:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:22 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:22.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:22 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:22 smithi184 ceph-mon[33536]: osdmap e396: 8 total, 8 up, 8 in 2024-02-14T17:50:22.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:22 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:23.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:23 smithi178 ceph-mon[39811]: osdmap e397: 8 total, 8 up, 8 in 2024-02-14T17:50:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:23 smithi178 ceph-mon[39811]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-14T17:50:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:23 smithi178 ceph-mon[30321]: osdmap e397: 8 total, 8 up, 8 in 2024-02-14T17:50:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:23 smithi178 ceph-mon[30321]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-14T17:50:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:23 smithi184 ceph-mon[33536]: osdmap e397: 8 total, 8 up, 8 in 2024-02-14T17:50:23.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:23 smithi184 ceph-mon[33536]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-14T17:50:24.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:24 smithi178 ceph-mon[39811]: osdmap e398: 8 total, 8 up, 8 in 2024-02-14T17:50:24.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:24 smithi178 ceph-mon[30321]: osdmap e398: 8 total, 8 up, 8 in 2024-02-14T17:50:24.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:24 smithi184 ceph-mon[33536]: osdmap e398: 8 total, 8 up, 8 in 2024-02-14T17:50:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:25 smithi184 ceph-mon[33536]: osdmap e399: 8 total, 8 up, 8 in 2024-02-14T17:50:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:25 smithi184 ceph-mon[33536]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-14T17:50:25.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:25 smithi178 ceph-mon[39811]: osdmap e399: 8 total, 8 up, 8 in 2024-02-14T17:50:25.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:25 smithi178 ceph-mon[39811]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-14T17:50:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:25 smithi178 ceph-mon[30321]: osdmap e399: 8 total, 8 up, 8 in 2024-02-14T17:50:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:25 smithi178 ceph-mon[30321]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-14T17:50:26.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:50:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:50:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:27.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:27 smithi178 ceph-mon[39811]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-14T17:50:27.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:27 smithi178 ceph-mon[30321]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-14T17:50:27.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:27 smithi184 ceph-mon[33536]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-14T17:50:28.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:28 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:28.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:28 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:28.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:28 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:29.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:29 smithi178 ceph-mon[39811]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 992 B/s wr, 3 op/s 2024-02-14T17:50:29.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:29 smithi178 ceph-mon[30321]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 992 B/s wr, 3 op/s 2024-02-14T17:50:29.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:29 smithi184 ceph-mon[33536]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 992 B/s wr, 3 op/s 2024-02-14T17:50:31.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:31 smithi178 ceph-mon[39811]: pgmap v564: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:50:31.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:31 smithi178 ceph-mon[39811]: osdmap e400: 8 total, 8 up, 8 in 2024-02-14T17:50:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:31 smithi178 ceph-mon[30321]: pgmap v564: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:50:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:31 smithi178 ceph-mon[30321]: osdmap e400: 8 total, 8 up, 8 in 2024-02-14T17:50:31.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:50:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:50:31.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:31 smithi184 ceph-mon[33536]: pgmap v564: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:50:31.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:31 smithi184 ceph-mon[33536]: osdmap e400: 8 total, 8 up, 8 in 2024-02-14T17:50:32.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:32 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:32.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:32 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:32.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:32 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:33.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:33 smithi178 ceph-mon[39811]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 751 B/s wr, 3 op/s 2024-02-14T17:50:33.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:33 smithi178 ceph-mon[30321]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 751 B/s wr, 3 op/s 2024-02-14T17:50:33.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:33 smithi184 ceph-mon[33536]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 751 B/s wr, 3 op/s 2024-02-14T17:50:34.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:34 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:50:34.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:34 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:50:34.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:34 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:50:34.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:34 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:50:34.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:34 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:50:34.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:34 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:50:34.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:34 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:50:34.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:34 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:50:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:34 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:50:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:34 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:50:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:34 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:50:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:34 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:50:35.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:35 smithi178 ceph-mon[30321]: pgmap v567: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-02-14T17:50:35.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:35 smithi178 ceph-mon[30321]: osdmap e401: 8 total, 8 up, 8 in 2024-02-14T17:50:35.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:35 smithi178 ceph-mon[39811]: pgmap v567: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-02-14T17:50:35.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:35 smithi178 ceph-mon[39811]: osdmap e401: 8 total, 8 up, 8 in 2024-02-14T17:50:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:35 smithi184 ceph-mon[33536]: pgmap v567: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-02-14T17:50:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:35 smithi184 ceph-mon[33536]: osdmap e401: 8 total, 8 up, 8 in 2024-02-14T17:50:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:50:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:50:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:36 smithi184 ceph-mon[33536]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:36.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:36 smithi178 ceph-mon[39811]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:36 smithi178 ceph-mon[30321]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:39.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:39 smithi178 ceph-mon[30321]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:39.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:39 smithi178 ceph-mon[39811]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:39.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:39 smithi184 ceph-mon[33536]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:41.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:41 smithi178 ceph-mon[39811]: pgmap v571: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:41.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:41 smithi178 ceph-mon[39811]: osdmap e402: 8 total, 8 up, 8 in 2024-02-14T17:50:41.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:41 smithi178 ceph-mon[30321]: pgmap v571: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:41.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:41 smithi178 ceph-mon[30321]: osdmap e402: 8 total, 8 up, 8 in 2024-02-14T17:50:41.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:50:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:50:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:41 smithi184 ceph-mon[33536]: pgmap v571: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:41 smithi184 ceph-mon[33536]: osdmap e402: 8 total, 8 up, 8 in 2024-02-14T17:50:42.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:42 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:42.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:42 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:42 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:43 smithi178 ceph-mon[30321]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:43.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:43 smithi178 ceph-mon[39811]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:43.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:43.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:43 smithi184 ceph-mon[33536]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:43.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:45.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:45 smithi178 ceph-mon[30321]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:45.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:45 smithi178 ceph-mon[39811]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:45.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:45 smithi184 ceph-mon[33536]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:46.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:50:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:50:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:46.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:46 smithi178 ceph-mon[39811]: osdmap e403: 8 total, 8 up, 8 in 2024-02-14T17:50:46.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:46 smithi178 ceph-mon[30321]: osdmap e403: 8 total, 8 up, 8 in 2024-02-14T17:50:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:46 smithi184 ceph-mon[33536]: osdmap e403: 8 total, 8 up, 8 in 2024-02-14T17:50:47.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:47 smithi178 ceph-mon[39811]: pgmap v576: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:50:47.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:47 smithi178 ceph-mon[30321]: pgmap v576: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:50:47.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:47 smithi184 ceph-mon[33536]: pgmap v576: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:50:49.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:49 smithi184 ceph-mon[33536]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:50:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:49 smithi178 ceph-mon[39811]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:50:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:49 smithi178 ceph-mon[30321]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:50:51.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:51 smithi184 ceph-mon[33536]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:50:51.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:50:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:50:51.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:51 smithi178 ceph-mon[39811]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:50:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:51 smithi178 ceph-mon[30321]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:50:52.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:52 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:52.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:52 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:52 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:53.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:53 smithi184 ceph-mon[33536]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:50:53.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:53 smithi178 ceph-mon[39811]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:50:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:53 smithi178 ceph-mon[30321]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:50:55.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:55 smithi178 ceph-mon[39811]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:50:55.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:55 smithi178 ceph-mon[30321]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:50:55.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:55 smithi184 ceph-mon[33536]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:50:56.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:50:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:50:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:56 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-63"}]: dispatch 2024-02-14T17:50:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:56 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-63"}]: dispatch 2024-02-14T17:50:56.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:56 smithi184 ceph-mon[33536]: osdmap e404: 8 total, 8 up, 8 in 2024-02-14T17:50:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:56 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-63"}]: dispatch 2024-02-14T17:50:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:56 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-63"}]: dispatch 2024-02-14T17:50:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:56 smithi178 ceph-mon[30321]: osdmap e404: 8 total, 8 up, 8 in 2024-02-14T17:50:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:56 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:50:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-63"}]: dispatch 2024-02-14T17:50:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:56 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-63"}]: dispatch 2024-02-14T17:50:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:56 smithi178 ceph-mon[39811]: osdmap e404: 8 total, 8 up, 8 in 2024-02-14T17:50:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:57 smithi184 ceph-mon[33536]: pgmap v582: 196 pgs: 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:57 smithi184 ceph-mon[33536]: osdmap e405: 8 total, 8 up, 8 in 2024-02-14T17:50:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:57.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:57 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:57.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:57 smithi178 ceph-mon[39811]: pgmap v582: 196 pgs: 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:57.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:57 smithi178 ceph-mon[39811]: osdmap e405: 8 total, 8 up, 8 in 2024-02-14T17:50:57.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:57.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:57 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:57 smithi178 ceph-mon[30321]: pgmap v582: 196 pgs: 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:57 smithi178 ceph-mon[30321]: osdmap e405: 8 total, 8 up, 8 in 2024-02-14T17:50:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:57 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:58.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:58 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:58 smithi184 ceph-mon[33536]: osdmap e406: 8 total, 8 up, 8 in 2024-02-14T17:50:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:58 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:58 smithi184 ceph-mon[33536]: pgmap v585: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:58.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:58 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:58.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[39811]: osdmap e406: 8 total, 8 up, 8 in 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[39811]: pgmap v585: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[30321]: osdmap e406: 8 total, 8 up, 8 in 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[30321]: pgmap v585: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:58 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:59.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:59 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:50:59.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:50:59 smithi184 ceph-mon[33536]: osdmap e407: 8 total, 8 up, 8 in 2024-02-14T17:50:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:59 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:50:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:50:59 smithi178 ceph-mon[39811]: osdmap e407: 8 total, 8 up, 8 in 2024-02-14T17:50:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:59 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:50:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:50:59 smithi178 ceph-mon[30321]: osdmap e407: 8 total, 8 up, 8 in 2024-02-14T17:51:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:00 smithi184 ceph-mon[33536]: osdmap e408: 8 total, 8 up, 8 in 2024-02-14T17:51:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:00 smithi184 ceph-mon[33536]: pgmap v588: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:00 smithi178 ceph-mon[39811]: osdmap e408: 8 total, 8 up, 8 in 2024-02-14T17:51:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:00 smithi178 ceph-mon[39811]: pgmap v588: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:00 smithi178 ceph-mon[30321]: osdmap e408: 8 total, 8 up, 8 in 2024-02-14T17:51:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:00 smithi178 ceph-mon[30321]: pgmap v588: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:01.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:01 smithi184 ceph-mon[33536]: osdmap e409: 8 total, 8 up, 8 in 2024-02-14T17:51:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:01 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:01.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:51:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:51:01.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:01 smithi178 ceph-mon[39811]: osdmap e409: 8 total, 8 up, 8 in 2024-02-14T17:51:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:01 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:01 smithi178 ceph-mon[30321]: osdmap e409: 8 total, 8 up, 8 in 2024-02-14T17:51:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:01 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:02 smithi184 ceph-mon[33536]: osdmap e410: 8 total, 8 up, 8 in 2024-02-14T17:51:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:02 smithi184 ceph-mon[33536]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-14T17:51:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:02 smithi178 ceph-mon[39811]: osdmap e410: 8 total, 8 up, 8 in 2024-02-14T17:51:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:02 smithi178 ceph-mon[39811]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-14T17:51:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:02 smithi178 ceph-mon[30321]: osdmap e410: 8 total, 8 up, 8 in 2024-02-14T17:51:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:02 smithi178 ceph-mon[30321]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-14T17:51:05.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:05 smithi178 ceph-mon[30321]: pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-02-14T17:51:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:05 smithi178 ceph-mon[39811]: pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-02-14T17:51:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:05 smithi184 ceph-mon[33536]: pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-02-14T17:51:06.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:51:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:51:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:07.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:07 smithi178 ceph-mon[30321]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-14T17:51:07.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:07 smithi178 ceph-mon[39811]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-14T17:51:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:07 smithi184 ceph-mon[33536]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-14T17:51:09.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:09 smithi178 ceph-mon[39811]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-14T17:51:09.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:09 smithi178 ceph-mon[30321]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-14T17:51:09.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:09 smithi184 ceph-mon[33536]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-14T17:51:11.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:11 smithi178 ceph-mon[39811]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 730 B/s wr, 3 op/s 2024-02-14T17:51:11.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:11 smithi178 ceph-mon[39811]: osdmap e411: 8 total, 8 up, 8 in 2024-02-14T17:51:11.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:11 smithi178 ceph-mon[30321]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 730 B/s wr, 3 op/s 2024-02-14T17:51:11.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:11 smithi178 ceph-mon[30321]: osdmap e411: 8 total, 8 up, 8 in 2024-02-14T17:51:11.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:51:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:51:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:11 smithi184 ceph-mon[33536]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 730 B/s wr, 3 op/s 2024-02-14T17:51:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:11 smithi184 ceph-mon[33536]: osdmap e411: 8 total, 8 up, 8 in 2024-02-14T17:51:12.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:12 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:12.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:12.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:12 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:12.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-65"}]: dispatch 2024-02-14T17:51:12.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:12 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-65"}]: dispatch 2024-02-14T17:51:12.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:12 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:12.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:12.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:12 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:12.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-65"}]: dispatch 2024-02-14T17:51:12.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:12 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-65"}]: dispatch 2024-02-14T17:51:12.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:12 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:12.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:12.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:12 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:12.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-65"}]: dispatch 2024-02-14T17:51:12.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:12 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-65"}]: dispatch 2024-02-14T17:51:13.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:13 smithi178 ceph-mon[39811]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 204 B/s wr, 2 op/s 2024-02-14T17:51:13.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:13 smithi178 ceph-mon[39811]: osdmap e412: 8 total, 8 up, 8 in 2024-02-14T17:51:13.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:13.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:13 smithi178 ceph-mon[30321]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 204 B/s wr, 2 op/s 2024-02-14T17:51:13.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:13 smithi178 ceph-mon[30321]: osdmap e412: 8 total, 8 up, 8 in 2024-02-14T17:51:13.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:13.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:13 smithi184 ceph-mon[33536]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 204 B/s wr, 2 op/s 2024-02-14T17:51:13.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:13 smithi184 ceph-mon[33536]: osdmap e412: 8 total, 8 up, 8 in 2024-02-14T17:51:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:14.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:14 smithi184 ceph-mon[33536]: osdmap e413: 8 total, 8 up, 8 in 2024-02-14T17:51:14.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:14.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:14 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:14.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:14 smithi178 ceph-mon[39811]: osdmap e413: 8 total, 8 up, 8 in 2024-02-14T17:51:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:14 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:14 smithi178 ceph-mon[30321]: osdmap e413: 8 total, 8 up, 8 in 2024-02-14T17:51:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:14 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:15.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:15 smithi178 ceph-mon[39811]: pgmap v600: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:15.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:15 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:15.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:15.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:15 smithi178 ceph-mon[39811]: osdmap e414: 8 total, 8 up, 8 in 2024-02-14T17:51:15.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:15.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:15 smithi178 ceph-mon[30321]: pgmap v600: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:15.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:15 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:15.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:15.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:15 smithi178 ceph-mon[30321]: osdmap e414: 8 total, 8 up, 8 in 2024-02-14T17:51:15.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:15.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:15 smithi184 ceph-mon[33536]: pgmap v600: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:15.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:15 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:15.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:15.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:15 smithi184 ceph-mon[33536]: osdmap e414: 8 total, 8 up, 8 in 2024-02-14T17:51:15.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:16.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:51:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:51:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:16.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:16 smithi184 ceph-mon[33536]: osdmap e415: 8 total, 8 up, 8 in 2024-02-14T17:51:16.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:16.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:16.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-67"}]: dispatch 2024-02-14T17:51:16.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-67"}]: dispatch 2024-02-14T17:51:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:16 smithi178 ceph-mon[30321]: osdmap e415: 8 total, 8 up, 8 in 2024-02-14T17:51:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-67"}]: dispatch 2024-02-14T17:51:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-67"}]: dispatch 2024-02-14T17:51:16.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:16 smithi178 ceph-mon[39811]: osdmap e415: 8 total, 8 up, 8 in 2024-02-14T17:51:16.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:16.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:16.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-67"}]: dispatch 2024-02-14T17:51:16.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-67"}]: dispatch 2024-02-14T17:51:17.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:17 smithi184 ceph-mon[33536]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:17.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:17 smithi184 ceph-mon[33536]: osdmap e416: 8 total, 8 up, 8 in 2024-02-14T17:51:17.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:17 smithi178 ceph-mon[30321]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:17.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:17 smithi178 ceph-mon[30321]: osdmap e416: 8 total, 8 up, 8 in 2024-02-14T17:51:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:17 smithi178 ceph-mon[39811]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:17 smithi178 ceph-mon[39811]: osdmap e416: 8 total, 8 up, 8 in 2024-02-14T17:51:18.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:18 smithi184 ceph-mon[33536]: osdmap e417: 8 total, 8 up, 8 in 2024-02-14T17:51:18.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:18.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:18 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:18.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:18 smithi184 ceph-mon[33536]: pgmap v606: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:51:18.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:18 smithi178 ceph-mon[39811]: osdmap e417: 8 total, 8 up, 8 in 2024-02-14T17:51:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:18 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:18 smithi178 ceph-mon[39811]: pgmap v606: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:51:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:18 smithi178 ceph-mon[30321]: osdmap e417: 8 total, 8 up, 8 in 2024-02-14T17:51:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:18 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:18 smithi178 ceph-mon[30321]: pgmap v606: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:51:19.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:19.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:19 smithi184 ceph-mon[33536]: osdmap e418: 8 total, 8 up, 8 in 2024-02-14T17:51:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-69"}]: dispatch 2024-02-14T17:51:19.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-69"}]: dispatch 2024-02-14T17:51:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[39811]: osdmap e418: 8 total, 8 up, 8 in 2024-02-14T17:51:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-69"}]: dispatch 2024-02-14T17:51:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-69"}]: dispatch 2024-02-14T17:51:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[30321]: osdmap e418: 8 total, 8 up, 8 in 2024-02-14T17:51:19.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-69"}]: dispatch 2024-02-14T17:51:19.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-69"}]: dispatch 2024-02-14T17:51:20.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:20 smithi184 ceph-mon[33536]: osdmap e419: 8 total, 8 up, 8 in 2024-02-14T17:51:20.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:20 smithi184 ceph-mon[33536]: pgmap v609: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:20.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:20 smithi178 ceph-mon[39811]: osdmap e419: 8 total, 8 up, 8 in 2024-02-14T17:51:20.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:20 smithi178 ceph-mon[39811]: pgmap v609: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:20 smithi178 ceph-mon[30321]: osdmap e419: 8 total, 8 up, 8 in 2024-02-14T17:51:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:20 smithi178 ceph-mon[30321]: pgmap v609: 196 pgs: 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:21.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:51:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:51:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:21 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:21 smithi184 ceph-mon[33536]: osdmap e420: 8 total, 8 up, 8 in 2024-02-14T17:51:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:21 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:21.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:21 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:21.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:21 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:21.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:21 smithi178 ceph-mon[30321]: osdmap e420: 8 total, 8 up, 8 in 2024-02-14T17:51:21.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:21.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:21 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:21.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:21 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:21 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:21 smithi178 ceph-mon[39811]: osdmap e420: 8 total, 8 up, 8 in 2024-02-14T17:51:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:21 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:21.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:21 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:22.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:22 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:22.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:22 smithi184 ceph-mon[33536]: osdmap e421: 8 total, 8 up, 8 in 2024-02-14T17:51:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:51:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:22 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:51:22.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:22 smithi184 ceph-mon[33536]: pgmap v612: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:51:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[30321]: osdmap e421: 8 total, 8 up, 8 in 2024-02-14T17:51:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:51:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:51:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[30321]: pgmap v612: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:51:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[39811]: osdmap e421: 8 total, 8 up, 8 in 2024-02-14T17:51:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:51:22.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:51:22.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:22 smithi178 ceph-mon[39811]: pgmap v612: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:51:23.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:23 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:51:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:23 smithi184 ceph-mon[33536]: osdmap e422: 8 total, 8 up, 8 in 2024-02-14T17:51:23.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:23 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:51:23.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:23 smithi178 ceph-mon[30321]: osdmap e422: 8 total, 8 up, 8 in 2024-02-14T17:51:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:23 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:51:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:23 smithi178 ceph-mon[39811]: osdmap e422: 8 total, 8 up, 8 in 2024-02-14T17:51:24.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:24 smithi184 ceph-mon[33536]: osdmap e423: 8 total, 8 up, 8 in 2024-02-14T17:51:24.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:24 smithi184 ceph-mon[33536]: pgmap v615: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:51:24.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:24 smithi178 ceph-mon[39811]: osdmap e423: 8 total, 8 up, 8 in 2024-02-14T17:51:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:24 smithi178 ceph-mon[39811]: pgmap v615: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:51:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:24 smithi178 ceph-mon[30321]: osdmap e423: 8 total, 8 up, 8 in 2024-02-14T17:51:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:24 smithi178 ceph-mon[30321]: pgmap v615: 228 pgs: 28 creating+peering, 4 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:51:26.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:51:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:51:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:27.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:27 smithi178 ceph-mon[39811]: pgmap v616: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-02-14T17:51:27.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:27 smithi178 ceph-mon[30321]: pgmap v616: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-02-14T17:51:27.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:27 smithi184 ceph-mon[33536]: pgmap v616: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-02-14T17:51:28.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:28 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:28.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:28 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:28.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:28 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:29 smithi178 ceph-mon[39811]: pgmap v617: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 892 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-14T17:51:29.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:29 smithi178 ceph-mon[30321]: pgmap v617: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 892 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-14T17:51:29.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:29 smithi184 ceph-mon[33536]: pgmap v617: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 892 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-14T17:51:31.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:31 smithi184 ceph-mon[33536]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-14T17:51:31.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:51:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:51:31.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:31 smithi178 ceph-mon[30321]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-14T17:51:31.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:31 smithi178 ceph-mon[39811]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-14T17:51:32.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:32 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:32 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:32 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:33.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:33 smithi184 ceph-mon[33536]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-14T17:51:33.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:33 smithi178 ceph-mon[39811]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-14T17:51:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:33 smithi178 ceph-mon[30321]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-14T17:51:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:34 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:51:34.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:34 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:51:34.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:34 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:51:35.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:35 smithi178 ceph-mon[30321]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:51:35.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:35 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:51:35.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:35 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:51:35.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:35 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:51:35.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:35 smithi178 ceph-mon[39811]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:51:35.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:35 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:51:35.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:35 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:51:35.690 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:35 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:51:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:35 smithi184 ceph-mon[33536]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:51:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:35 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:51:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:35 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:51:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:35 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:51:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:51:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:51:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:37.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:37 smithi184 ceph-mon[33536]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-14T17:51:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:37 smithi178 ceph-mon[39811]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-14T17:51:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:37 smithi178 ceph-mon[30321]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-14T17:51:39.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:39 smithi184 ceph-mon[33536]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:39.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:39 smithi178 ceph-mon[39811]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:39 smithi178 ceph-mon[30321]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:41.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:41 smithi184 ceph-mon[33536]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:41.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:51:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:51:41.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:41 smithi178 ceph-mon[39811]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:41 smithi178 ceph-mon[30321]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:42 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:42.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:42 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:42 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:43.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:43 smithi184 ceph-mon[33536]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:43.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:43.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:43 smithi178 ceph-mon[39811]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:43 smithi178 ceph-mon[30321]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:44 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:44.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-71"}]: dispatch 2024-02-14T17:51:44.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:44 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-71"}]: dispatch 2024-02-14T17:51:44.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:44 smithi184 ceph-mon[33536]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:44.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:44 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-71"}]: dispatch 2024-02-14T17:51:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:44 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-71"}]: dispatch 2024-02-14T17:51:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:44 smithi178 ceph-mon[30321]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:44 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-71"}]: dispatch 2024-02-14T17:51:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:44 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-71"}]: dispatch 2024-02-14T17:51:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:44 smithi178 ceph-mon[39811]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:51:45.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:45 smithi178 ceph-mon[39811]: osdmap e424: 8 total, 8 up, 8 in 2024-02-14T17:51:45.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:45 smithi178 ceph-mon[30321]: osdmap e424: 8 total, 8 up, 8 in 2024-02-14T17:51:45.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:45 smithi184 ceph-mon[33536]: osdmap e424: 8 total, 8 up, 8 in 2024-02-14T17:51:46.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:51:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:51:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:46 smithi184 ceph-mon[33536]: osdmap e425: 8 total, 8 up, 8 in 2024-02-14T17:51:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:46 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:46 smithi184 ceph-mon[33536]: pgmap v628: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:46 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:46 smithi178 ceph-mon[30321]: osdmap e425: 8 total, 8 up, 8 in 2024-02-14T17:51:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:46 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:46 smithi178 ceph-mon[30321]: pgmap v628: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:46 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:46 smithi178 ceph-mon[39811]: osdmap e425: 8 total, 8 up, 8 in 2024-02-14T17:51:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:46 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:46 smithi178 ceph-mon[39811]: pgmap v628: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:46 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:47.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:47 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:47.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:47 smithi184 ceph-mon[33536]: osdmap e426: 8 total, 8 up, 8 in 2024-02-14T17:51:47.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:47.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:47 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:47 smithi178 ceph-mon[39811]: osdmap e426: 8 total, 8 up, 8 in 2024-02-14T17:51:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:47 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:47 smithi178 ceph-mon[30321]: osdmap e426: 8 total, 8 up, 8 in 2024-02-14T17:51:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:51:48.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:48 smithi184 ceph-mon[33536]: osdmap e427: 8 total, 8 up, 8 in 2024-02-14T17:51:48.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:48 smithi184 ceph-mon[33536]: pgmap v631: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:48.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:48 smithi178 ceph-mon[39811]: osdmap e427: 8 total, 8 up, 8 in 2024-02-14T17:51:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:48 smithi178 ceph-mon[39811]: pgmap v631: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:48 smithi178 ceph-mon[30321]: osdmap e427: 8 total, 8 up, 8 in 2024-02-14T17:51:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:48 smithi178 ceph-mon[30321]: pgmap v631: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:49.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:49 smithi184 ceph-mon[33536]: osdmap e428: 8 total, 8 up, 8 in 2024-02-14T17:51:49.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:49 smithi178 ceph-mon[39811]: osdmap e428: 8 total, 8 up, 8 in 2024-02-14T17:51:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:49 smithi178 ceph-mon[30321]: osdmap e428: 8 total, 8 up, 8 in 2024-02-14T17:51:50.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:50 smithi184 ceph-mon[33536]: pgmap v633: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-14T17:51:50.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:50 smithi178 ceph-mon[39811]: pgmap v633: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-14T17:51:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:50 smithi178 ceph-mon[30321]: pgmap v633: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-14T17:51:51.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:51:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:51:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:51 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:51.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:51 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:51.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:51 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:52.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:52 smithi184 ceph-mon[33536]: pgmap v634: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-14T17:51:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:52 smithi178 ceph-mon[39811]: pgmap v634: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-14T17:51:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:52 smithi178 ceph-mon[30321]: pgmap v634: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-14T17:51:55.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:55 smithi178 ceph-mon[39811]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 907 B/s wr, 3 op/s 2024-02-14T17:51:55.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:55 smithi178 ceph-mon[30321]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 907 B/s wr, 3 op/s 2024-02-14T17:51:55.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:55 smithi184 ceph-mon[33536]: pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 907 B/s wr, 3 op/s 2024-02-14T17:51:56.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:51:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:51:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:56 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:56.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:56 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:56 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:57 smithi184 ceph-mon[33536]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 805 B/s wr, 3 op/s 2024-02-14T17:51:57.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:57 smithi178 ceph-mon[39811]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 805 B/s wr, 3 op/s 2024-02-14T17:51:57.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:57 smithi178 ceph-mon[30321]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 805 B/s wr, 3 op/s 2024-02-14T17:51:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:59 smithi184 ceph-mon[33536]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-02-14T17:51:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:59 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:59 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:51:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-73"}]: dispatch 2024-02-14T17:51:59.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:51:59 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-73"}]: dispatch 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[39811]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-73"}]: dispatch 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-73"}]: dispatch 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[30321]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-73"}]: dispatch 2024-02-14T17:52:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:51:59 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-73"}]: dispatch 2024-02-14T17:52:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:00 smithi184 ceph-mon[33536]: osdmap e429: 8 total, 8 up, 8 in 2024-02-14T17:52:00.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:00 smithi184 ceph-mon[33536]: pgmap v639: 196 pgs: 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s 2024-02-14T17:52:00.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:00 smithi184 ceph-mon[33536]: osdmap e430: 8 total, 8 up, 8 in 2024-02-14T17:52:00.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:00.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:00 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:01.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:00 smithi178 ceph-mon[39811]: osdmap e429: 8 total, 8 up, 8 in 2024-02-14T17:52:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:00 smithi178 ceph-mon[39811]: pgmap v639: 196 pgs: 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s 2024-02-14T17:52:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:00 smithi178 ceph-mon[39811]: osdmap e430: 8 total, 8 up, 8 in 2024-02-14T17:52:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:00 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:00 smithi178 ceph-mon[30321]: osdmap e429: 8 total, 8 up, 8 in 2024-02-14T17:52:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:00 smithi178 ceph-mon[30321]: pgmap v639: 196 pgs: 196 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s 2024-02-14T17:52:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:00 smithi178 ceph-mon[30321]: osdmap e430: 8 total, 8 up, 8 in 2024-02-14T17:52:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:00 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:01.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:52:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:52:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:02 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:02 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:02 smithi184 ceph-mon[33536]: osdmap e431: 8 total, 8 up, 8 in 2024-02-14T17:52:02.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:02 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:02 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:02 smithi178 ceph-mon[39811]: osdmap e431: 8 total, 8 up, 8 in 2024-02-14T17:52:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:02 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:02 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:02 smithi178 ceph-mon[30321]: osdmap e431: 8 total, 8 up, 8 in 2024-02-14T17:52:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:03 smithi184 ceph-mon[33536]: pgmap v642: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:03 smithi184 ceph-mon[33536]: osdmap e432: 8 total, 8 up, 8 in 2024-02-14T17:52:03.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:03 smithi178 ceph-mon[39811]: pgmap v642: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:03 smithi178 ceph-mon[39811]: osdmap e432: 8 total, 8 up, 8 in 2024-02-14T17:52:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:03 smithi178 ceph-mon[30321]: pgmap v642: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:03 smithi178 ceph-mon[30321]: osdmap e432: 8 total, 8 up, 8 in 2024-02-14T17:52:04.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:04 smithi184 ceph-mon[33536]: osdmap e433: 8 total, 8 up, 8 in 2024-02-14T17:52:04.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:04.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:04 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:04.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-75"}]: dispatch 2024-02-14T17:52:04.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:04 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-75"}]: dispatch 2024-02-14T17:52:04.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:04 smithi178 ceph-mon[39811]: osdmap e433: 8 total, 8 up, 8 in 2024-02-14T17:52:04.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:04 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-75"}]: dispatch 2024-02-14T17:52:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:04 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-75"}]: dispatch 2024-02-14T17:52:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:04 smithi178 ceph-mon[30321]: osdmap e433: 8 total, 8 up, 8 in 2024-02-14T17:52:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:04 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-75"}]: dispatch 2024-02-14T17:52:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:04 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-75"}]: dispatch 2024-02-14T17:52:05.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:05 smithi178 ceph-mon[39811]: pgmap v645: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:05.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:05 smithi178 ceph-mon[39811]: osdmap e434: 8 total, 8 up, 8 in 2024-02-14T17:52:05.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:05 smithi178 ceph-mon[30321]: pgmap v645: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:05.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:05 smithi178 ceph-mon[30321]: osdmap e434: 8 total, 8 up, 8 in 2024-02-14T17:52:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:05 smithi184 ceph-mon[33536]: pgmap v645: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:05 smithi184 ceph-mon[33536]: osdmap e434: 8 total, 8 up, 8 in 2024-02-14T17:52:06.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:52:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:52:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:06 smithi184 ceph-mon[33536]: osdmap e435: 8 total, 8 up, 8 in 2024-02-14T17:52:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:06 smithi184 ceph-mon[33536]: pgmap v648: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 767 B/s wr, 29 op/s 2024-02-14T17:52:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:06 smithi178 ceph-mon[39811]: osdmap e435: 8 total, 8 up, 8 in 2024-02-14T17:52:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:06 smithi178 ceph-mon[39811]: pgmap v648: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 767 B/s wr, 29 op/s 2024-02-14T17:52:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:06 smithi178 ceph-mon[30321]: osdmap e435: 8 total, 8 up, 8 in 2024-02-14T17:52:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:06 smithi178 ceph-mon[30321]: pgmap v648: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 767 B/s wr, 29 op/s 2024-02-14T17:52:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:07 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:07 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:07 smithi184 ceph-mon[33536]: osdmap e436: 8 total, 8 up, 8 in 2024-02-14T17:52:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:07.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:07 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:07 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:07 smithi178 ceph-mon[39811]: osdmap e436: 8 total, 8 up, 8 in 2024-02-14T17:52:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:07 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:07 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:07 smithi178 ceph-mon[30321]: osdmap e436: 8 total, 8 up, 8 in 2024-02-14T17:52:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:08 smithi184 ceph-mon[33536]: osdmap e437: 8 total, 8 up, 8 in 2024-02-14T17:52:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:08 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-77"}]: dispatch 2024-02-14T17:52:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:08 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-77"}]: dispatch 2024-02-14T17:52:08.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:08 smithi184 ceph-mon[33536]: pgmap v651: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 767 B/s wr, 29 op/s 2024-02-14T17:52:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[39811]: osdmap e437: 8 total, 8 up, 8 in 2024-02-14T17:52:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-77"}]: dispatch 2024-02-14T17:52:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-77"}]: dispatch 2024-02-14T17:52:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[39811]: pgmap v651: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 767 B/s wr, 29 op/s 2024-02-14T17:52:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[30321]: osdmap e437: 8 total, 8 up, 8 in 2024-02-14T17:52:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-77"}]: dispatch 2024-02-14T17:52:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-77"}]: dispatch 2024-02-14T17:52:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:08 smithi178 ceph-mon[30321]: pgmap v651: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 767 B/s wr, 29 op/s 2024-02-14T17:52:09.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:09 smithi184 ceph-mon[33536]: osdmap e438: 8 total, 8 up, 8 in 2024-02-14T17:52:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:09 smithi178 ceph-mon[39811]: osdmap e438: 8 total, 8 up, 8 in 2024-02-14T17:52:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:09 smithi178 ceph-mon[30321]: osdmap e438: 8 total, 8 up, 8 in 2024-02-14T17:52:10.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:10 smithi184 ceph-mon[33536]: osdmap e439: 8 total, 8 up, 8 in 2024-02-14T17:52:10.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:10.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:10 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:10.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:10 smithi184 ceph-mon[33536]: pgmap v654: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:10 smithi178 ceph-mon[30321]: osdmap e439: 8 total, 8 up, 8 in 2024-02-14T17:52:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:10 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:10 smithi178 ceph-mon[30321]: pgmap v654: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:10 smithi178 ceph-mon[39811]: osdmap e439: 8 total, 8 up, 8 in 2024-02-14T17:52:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:10 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:10 smithi178 ceph-mon[39811]: pgmap v654: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:11.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:52:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:52:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:11 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:11 smithi184 ceph-mon[33536]: osdmap e440: 8 total, 8 up, 8 in 2024-02-14T17:52:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:11.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:11 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:11.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:11 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:11 smithi178 ceph-mon[30321]: osdmap e440: 8 total, 8 up, 8 in 2024-02-14T17:52:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:11 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:11 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:11 smithi178 ceph-mon[39811]: osdmap e440: 8 total, 8 up, 8 in 2024-02-14T17:52:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:11 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:12.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:12 smithi184 ceph-mon[33536]: osdmap e441: 8 total, 8 up, 8 in 2024-02-14T17:52:12.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:12 smithi184 ceph-mon[33536]: pgmap v657: 228 pgs: 3 creating+activating, 6 creating+peering, 219 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 1.2 KiB/s wr, 155 op/s 2024-02-14T17:52:12.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:12 smithi178 ceph-mon[39811]: osdmap e441: 8 total, 8 up, 8 in 2024-02-14T17:52:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:12 smithi178 ceph-mon[39811]: pgmap v657: 228 pgs: 3 creating+activating, 6 creating+peering, 219 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 1.2 KiB/s wr, 155 op/s 2024-02-14T17:52:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:12 smithi178 ceph-mon[30321]: osdmap e441: 8 total, 8 up, 8 in 2024-02-14T17:52:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:12 smithi178 ceph-mon[30321]: pgmap v657: 228 pgs: 3 creating+activating, 6 creating+peering, 219 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 1.2 KiB/s wr, 155 op/s 2024-02-14T17:52:13.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:13 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:13.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:13 smithi184 ceph-mon[33536]: osdmap e442: 8 total, 8 up, 8 in 2024-02-14T17:52:13.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.11"}]: dispatch 2024-02-14T17:52:13.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:13 smithi184 ceph-mon[33536]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.11"}]: dispatch 2024-02-14T17:52:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:13 smithi184 ceph-mon[33536]: 305.11 deep-scrub starts 2024-02-14T17:52:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:13 smithi184 ceph-mon[33536]: 305.11 deep-scrub ok 2024-02-14T17:52:13.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[39811]: osdmap e442: 8 total, 8 up, 8 in 2024-02-14T17:52:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.11"}]: dispatch 2024-02-14T17:52:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[39811]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.11"}]: dispatch 2024-02-14T17:52:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[39811]: 305.11 deep-scrub starts 2024-02-14T17:52:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[39811]: 305.11 deep-scrub ok 2024-02-14T17:52:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[30321]: osdmap e442: 8 total, 8 up, 8 in 2024-02-14T17:52:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.11"}]: dispatch 2024-02-14T17:52:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[30321]: from='mon.? v1:172.21.15.178:0/2219313583' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.11"}]: dispatch 2024-02-14T17:52:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[30321]: 305.11 deep-scrub starts 2024-02-14T17:52:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[30321]: 305.11 deep-scrub ok 2024-02-14T17:52:13.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:14.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:14 smithi184 ceph-mon[33536]: pgmap v659: 228 pgs: 3 creating+activating, 6 creating+peering, 219 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 1.0 KiB/s wr, 127 op/s 2024-02-14T17:52:14.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:14 smithi178 ceph-mon[39811]: pgmap v659: 228 pgs: 3 creating+activating, 6 creating+peering, 219 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 1.0 KiB/s wr, 127 op/s 2024-02-14T17:52:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:14 smithi178 ceph-mon[30321]: pgmap v659: 228 pgs: 3 creating+activating, 6 creating+peering, 219 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 1.0 KiB/s wr, 127 op/s 2024-02-14T17:52:16.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:52:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:52:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:17.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:17 smithi184 ceph-mon[33536]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 1.2 KiB/s wr, 105 op/s 2024-02-14T17:52:17.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:17 smithi178 ceph-mon[30321]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 1.2 KiB/s wr, 105 op/s 2024-02-14T17:52:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:17 smithi178 ceph-mon[39811]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 1.2 KiB/s wr, 105 op/s 2024-02-14T17:52:19.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:19 smithi184 ceph-mon[33536]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 908 B/s wr, 60 op/s 2024-02-14T17:52:19.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:19 smithi178 ceph-mon[30321]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 908 B/s wr, 60 op/s 2024-02-14T17:52:19.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:19 smithi178 ceph-mon[39811]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 36 KiB/s rd, 908 B/s wr, 60 op/s 2024-02-14T17:52:21.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:21 smithi184 ceph-mon[33536]: pgmap v662: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 230 B/s wr, 1 op/s 2024-02-14T17:52:21.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:52:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:52:21.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:21 smithi178 ceph-mon[39811]: pgmap v662: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 230 B/s wr, 1 op/s 2024-02-14T17:52:21.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:21 smithi178 ceph-mon[30321]: pgmap v662: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 230 B/s wr, 1 op/s 2024-02-14T17:52:22.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:22 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:22.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:22 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:22 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:23.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:23 smithi184 ceph-mon[33536]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-02-14T17:52:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:23 smithi178 ceph-mon[39811]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-02-14T17:52:23.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:23 smithi178 ceph-mon[30321]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-02-14T17:52:25.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:25 smithi178 ceph-mon[39811]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 861 B/s rd, 172 B/s wr, 1 op/s 2024-02-14T17:52:25.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:25 smithi178 ceph-mon[30321]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 861 B/s rd, 172 B/s wr, 1 op/s 2024-02-14T17:52:25.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:25 smithi184 ceph-mon[33536]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 861 B/s rd, 172 B/s wr, 1 op/s 2024-02-14T17:52:26.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:52:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:52:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:26.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:26 smithi184 ceph-mon[33536]: pgmap v665: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-02-14T17:52:26.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:26 smithi178 ceph-mon[39811]: pgmap v665: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-02-14T17:52:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:26 smithi178 ceph-mon[30321]: pgmap v665: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-02-14T17:52:29.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:29 smithi184 ceph-mon[33536]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:52:29.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:29 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:29.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:29 smithi178 ceph-mon[39811]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:52:29.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:29 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:29.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:29 smithi178 ceph-mon[30321]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:52:29.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:29 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:31.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:31 smithi184 ceph-mon[33536]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:52:31.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:52:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:52:31.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:31 smithi178 ceph-mon[30321]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:52:31.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:31 smithi178 ceph-mon[39811]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:52:32.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:32 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:32.720 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 17:52:32 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T17:52:32.447+0000 7f65be193700 -1 snap_mapper.add_oid found existing snaps mapped on 305:8f254911:test-rados-api-smithi178-80903-80::foo:2, removing 2024-02-14T17:52:32.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:32 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:32 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:32.833 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 17:52:32 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0[44215]: 2024-02-14T17:52:32.446+0000 7efd32a4b700 -1 snap_mapper.add_oid found existing snaps mapped on 305:8f254911:test-rados-api-smithi178-80903-80::foo:2, removing 2024-02-14T17:52:32.833 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:52:32 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2[56763]: 2024-02-14T17:52:32.447+0000 7f0a32c14700 -1 snap_mapper.add_oid found existing snaps mapped on 305:8f254911:test-rados-api-smithi178-80903-80::foo:2, removing 2024-02-14T17:52:33.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:33 smithi184 ceph-mon[33536]: pgmap v668: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:33.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:33.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:33 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:33.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-79"}]: dispatch 2024-02-14T17:52:33.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:33 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-79"}]: dispatch 2024-02-14T17:52:33.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:33 smithi178 ceph-mon[30321]: pgmap v668: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:33 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-79"}]: dispatch 2024-02-14T17:52:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:33 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-79"}]: dispatch 2024-02-14T17:52:33.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:33 smithi178 ceph-mon[39811]: pgmap v668: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:33.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:33.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:33 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:33.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-79"}]: dispatch 2024-02-14T17:52:33.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:33 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-79"}]: dispatch 2024-02-14T17:52:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:34 smithi184 ceph-mon[33536]: osdmap e443: 8 total, 8 up, 8 in 2024-02-14T17:52:34.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:34 smithi178 ceph-mon[30321]: osdmap e443: 8 total, 8 up, 8 in 2024-02-14T17:52:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:34 smithi178 ceph-mon[39811]: osdmap e443: 8 total, 8 up, 8 in 2024-02-14T17:52:35.688 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:35 smithi178 ceph-mon[30321]: pgmap v670: 196 pgs: 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:52:35.688 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:35 smithi178 ceph-mon[30321]: osdmap e444: 8 total, 8 up, 8 in 2024-02-14T17:52:35.688 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:35.688 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:35 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:35.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:35 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:52:35.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:35 smithi178 ceph-mon[39811]: pgmap v670: 196 pgs: 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:52:35.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:35 smithi178 ceph-mon[39811]: osdmap e444: 8 total, 8 up, 8 in 2024-02-14T17:52:35.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:35.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:35 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:35.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:35 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:52:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:35 smithi184 ceph-mon[33536]: pgmap v670: 196 pgs: 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:52:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:35 smithi184 ceph-mon[33536]: osdmap e444: 8 total, 8 up, 8 in 2024-02-14T17:52:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:35 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:35 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:52:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:52:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:52:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:36 smithi184 ceph-mon[33536]: osdmap e445: 8 total, 8 up, 8 in 2024-02-14T17:52:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:36 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:52:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:52:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:52:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:36 smithi184 ceph-mon[33536]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-02-14T17:52:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[39811]: osdmap e445: 8 total, 8 up, 8 in 2024-02-14T17:52:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:52:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:52:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:52:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[39811]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-02-14T17:52:36.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:36.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[30321]: osdmap e445: 8 total, 8 up, 8 in 2024-02-14T17:52:36.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:36.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:36.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:36.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:52:36.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:52:36.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:52:36.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:36 smithi178 ceph-mon[30321]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-02-14T17:52:37.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:37 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:52:37.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:37 smithi184 ceph-mon[33536]: osdmap e446: 8 total, 8 up, 8 in 2024-02-14T17:52:37.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:37.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:37 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:37.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:37 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:52:37.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:37 smithi178 ceph-mon[39811]: osdmap e446: 8 total, 8 up, 8 in 2024-02-14T17:52:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:37 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:37 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:52:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:37 smithi178 ceph-mon[30321]: osdmap e446: 8 total, 8 up, 8 in 2024-02-14T17:52:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:37 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:38.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:38 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:52:38.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:38 smithi184 ceph-mon[33536]: osdmap e447: 8 total, 8 up, 8 in 2024-02-14T17:52:38.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:38.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:38 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:38.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:38 smithi184 ceph-mon[33536]: pgmap v676: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:38.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:38 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:52:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:38 smithi178 ceph-mon[39811]: osdmap e447: 8 total, 8 up, 8 in 2024-02-14T17:52:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:38 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:38 smithi178 ceph-mon[39811]: pgmap v676: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:38 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:52:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:38 smithi178 ceph-mon[30321]: osdmap e447: 8 total, 8 up, 8 in 2024-02-14T17:52:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:38 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:38 smithi178 ceph-mon[30321]: pgmap v676: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:39.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:39 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:52:39.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:39 smithi184 ceph-mon[33536]: osdmap e448: 8 total, 8 up, 8 in 2024-02-14T17:52:39.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:39.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:39 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:39.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:39 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:52:39.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:39 smithi178 ceph-mon[39811]: osdmap e448: 8 total, 8 up, 8 in 2024-02-14T17:52:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:39 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:39 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:52:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:39 smithi178 ceph-mon[30321]: osdmap e448: 8 total, 8 up, 8 in 2024-02-14T17:52:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:39 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:40.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:40 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:40 smithi184 ceph-mon[33536]: osdmap e449: 8 total, 8 up, 8 in 2024-02-14T17:52:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:52:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:40 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:52:40.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:40 smithi184 ceph-mon[33536]: pgmap v679: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:40 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:40 smithi178 ceph-mon[39811]: osdmap e449: 8 total, 8 up, 8 in 2024-02-14T17:52:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:52:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:40 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:52:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:40 smithi178 ceph-mon[39811]: pgmap v679: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:40 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:40 smithi178 ceph-mon[30321]: osdmap e449: 8 total, 8 up, 8 in 2024-02-14T17:52:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:52:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:40 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:52:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:40 smithi178 ceph-mon[30321]: pgmap v679: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:41.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:52:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:52:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:41 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-14T17:52:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:41 smithi184 ceph-mon[33536]: osdmap e450: 8 total, 8 up, 8 in 2024-02-14T17:52:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:52:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:41 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:52:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:41 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.3", "id": [7, 2]}]: dispatch 2024-02-14T17:52:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:41 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.3", "id": [7, 2]}]: dispatch 2024-02-14T17:52:41.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:41 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:41.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-14T17:52:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[39811]: osdmap e450: 8 total, 8 up, 8 in 2024-02-14T17:52:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:52:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:52:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.3", "id": [7, 2]}]: dispatch 2024-02-14T17:52:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.3", "id": [7, 2]}]: dispatch 2024-02-14T17:52:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-14T17:52:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[30321]: osdmap e450: 8 total, 8 up, 8 in 2024-02-14T17:52:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:52:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:52:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.3", "id": [7, 2]}]: dispatch 2024-02-14T17:52:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.3", "id": [7, 2]}]: dispatch 2024-02-14T17:52:41.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:41 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:42 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-14T17:52:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:42 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.3", "id": [7, 2]}]': finished 2024-02-14T17:52:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:42 smithi184 ceph-mon[33536]: osdmap e451: 8 total, 8 up, 8 in 2024-02-14T17:52:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:42 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:42 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:42.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:42 smithi184 ceph-mon[33536]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 8.2 KiB/s wr, 17 op/s 2024-02-14T17:52:42.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-14T17:52:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.3", "id": [7, 2]}]': finished 2024-02-14T17:52:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[39811]: osdmap e451: 8 total, 8 up, 8 in 2024-02-14T17:52:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[39811]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 8.2 KiB/s wr, 17 op/s 2024-02-14T17:52:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-14T17:52:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "306.3", "id": [7, 2]}]': finished 2024-02-14T17:52:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[30321]: osdmap e451: 8 total, 8 up, 8 in 2024-02-14T17:52:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:42 smithi178 ceph-mon[30321]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 8.2 KiB/s wr, 17 op/s 2024-02-14T17:52:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:43 smithi184 ceph-mon[33536]: osdmap e452: 8 total, 8 up, 8 in 2024-02-14T17:52:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-81"}]: dispatch 2024-02-14T17:52:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:43 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-81"}]: dispatch 2024-02-14T17:52:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[39811]: osdmap e452: 8 total, 8 up, 8 in 2024-02-14T17:52:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-81"}]: dispatch 2024-02-14T17:52:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-81"}]: dispatch 2024-02-14T17:52:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[30321]: osdmap e452: 8 total, 8 up, 8 in 2024-02-14T17:52:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-81"}]: dispatch 2024-02-14T17:52:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-81"}]: dispatch 2024-02-14T17:52:43.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:44 smithi184 ceph-mon[33536]: osdmap e453: 8 total, 8 up, 8 in 2024-02-14T17:52:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:44 smithi184 ceph-mon[33536]: pgmap v685: 196 pgs: 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-02-14T17:52:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:44 smithi178 ceph-mon[39811]: osdmap e453: 8 total, 8 up, 8 in 2024-02-14T17:52:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:44 smithi178 ceph-mon[39811]: pgmap v685: 196 pgs: 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-02-14T17:52:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:44 smithi178 ceph-mon[30321]: osdmap e453: 8 total, 8 up, 8 in 2024-02-14T17:52:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:44 smithi178 ceph-mon[30321]: pgmap v685: 196 pgs: 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-02-14T17:52:45.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:45 smithi184 ceph-mon[33536]: osdmap e454: 8 total, 8 up, 8 in 2024-02-14T17:52:45.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:45.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:45 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:45.733 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:52:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:52:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:45.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:45 smithi178 ceph-mon[39811]: osdmap e454: 8 total, 8 up, 8 in 2024-02-14T17:52:45.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:45.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:45 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:45.734 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:45 smithi178 ceph-mon[30321]: osdmap e454: 8 total, 8 up, 8 in 2024-02-14T17:52:45.734 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:45.734 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:45 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:46 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:46 smithi184 ceph-mon[33536]: osdmap e455: 8 total, 8 up, 8 in 2024-02-14T17:52:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:46.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:46.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:46 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:46.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:46 smithi184 ceph-mon[33536]: pgmap v688: 228 pgs: 6 creating+peering, 222 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 23 op/s 2024-02-14T17:52:46.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[39811]: osdmap e455: 8 total, 8 up, 8 in 2024-02-14T17:52:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[39811]: pgmap v688: 228 pgs: 6 creating+peering, 222 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 23 op/s 2024-02-14T17:52:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[30321]: osdmap e455: 8 total, 8 up, 8 in 2024-02-14T17:52:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:46 smithi178 ceph-mon[30321]: pgmap v688: 228 pgs: 6 creating+peering, 222 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 23 op/s 2024-02-14T17:52:47.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:47 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:52:47.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:47 smithi184 ceph-mon[33536]: osdmap e456: 8 total, 8 up, 8 in 2024-02-14T17:52:47.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:47.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:47 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:47.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:47 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:52:47.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:47 smithi178 ceph-mon[39811]: osdmap e456: 8 total, 8 up, 8 in 2024-02-14T17:52:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:47 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:47 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:52:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:47 smithi178 ceph-mon[30321]: osdmap e456: 8 total, 8 up, 8 in 2024-02-14T17:52:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:47 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:48.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:52:48.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:48 smithi184 ceph-mon[33536]: osdmap e457: 8 total, 8 up, 8 in 2024-02-14T17:52:48.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:48.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:48.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:48 smithi184 ceph-mon[33536]: pgmap v691: 228 pgs: 6 creating+peering, 222 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 6.0 KiB/s wr, 23 op/s 2024-02-14T17:52:48.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:52:48.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:48 smithi184 ceph-mon[33536]: osdmap e458: 8 total, 8 up, 8 in 2024-02-14T17:52:48.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:52:48.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[39811]: osdmap e457: 8 total, 8 up, 8 in 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[39811]: pgmap v691: 228 pgs: 6 creating+peering, 222 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 6.0 KiB/s wr, 23 op/s 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[39811]: osdmap e458: 8 total, 8 up, 8 in 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[30321]: osdmap e457: 8 total, 8 up, 8 in 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[30321]: pgmap v691: 228 pgs: 6 creating+peering, 222 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 6.0 KiB/s wr, 23 op/s 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:52:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:48 smithi178 ceph-mon[30321]: osdmap e458: 8 total, 8 up, 8 in 2024-02-14T17:52:49.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:49.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:49 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:49.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:49 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:49.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:49.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:49 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:49 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:49 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:49 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:50.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:50 smithi184 ceph-mon[33536]: osdmap e459: 8 total, 8 up, 8 in 2024-02-14T17:52:50.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:50 smithi184 ceph-mon[33536]: pgmap v694: 228 pgs: 6 creating+peering, 222 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:50.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:50 smithi178 ceph-mon[30321]: osdmap e459: 8 total, 8 up, 8 in 2024-02-14T17:52:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:50 smithi178 ceph-mon[30321]: pgmap v694: 228 pgs: 6 creating+peering, 222 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:50.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:50 smithi178 ceph-mon[39811]: osdmap e459: 8 total, 8 up, 8 in 2024-02-14T17:52:50.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:50 smithi178 ceph-mon[39811]: pgmap v694: 228 pgs: 6 creating+peering, 222 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:51.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:52:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:52:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:51 smithi184 ceph-mon[33536]: osdmap e460: 8 total, 8 up, 8 in 2024-02-14T17:52:51.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:51 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:51.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:51 smithi178 ceph-mon[39811]: osdmap e460: 8 total, 8 up, 8 in 2024-02-14T17:52:51.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:51 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:51 smithi178 ceph-mon[30321]: osdmap e460: 8 total, 8 up, 8 in 2024-02-14T17:52:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:51 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:52.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[30321]: osdmap e461: 8 total, 8 up, 8 in 2024-02-14T17:52:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-83"}]: dispatch 2024-02-14T17:52:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-83"}]: dispatch 2024-02-14T17:52:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[30321]: pgmap v697: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-14T17:52:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[39811]: osdmap e461: 8 total, 8 up, 8 in 2024-02-14T17:52:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-83"}]: dispatch 2024-02-14T17:52:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-83"}]: dispatch 2024-02-14T17:52:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:52 smithi178 ceph-mon[39811]: pgmap v697: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-14T17:52:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:52 smithi184 ceph-mon[33536]: osdmap e461: 8 total, 8 up, 8 in 2024-02-14T17:52:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:52 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-83"}]: dispatch 2024-02-14T17:52:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:52 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-83"}]: dispatch 2024-02-14T17:52:52.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:52 smithi184 ceph-mon[33536]: pgmap v697: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-14T17:52:53.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:53 smithi178 ceph-mon[30321]: osdmap e462: 8 total, 8 up, 8 in 2024-02-14T17:52:53.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:53 smithi178 ceph-mon[39811]: osdmap e462: 8 total, 8 up, 8 in 2024-02-14T17:52:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:53 smithi184 ceph-mon[33536]: osdmap e462: 8 total, 8 up, 8 in 2024-02-14T17:52:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:54 smithi178 ceph-mon[39811]: osdmap e463: 8 total, 8 up, 8 in 2024-02-14T17:52:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:54 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:54 smithi178 ceph-mon[39811]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:52:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:54 smithi178 ceph-mon[30321]: osdmap e463: 8 total, 8 up, 8 in 2024-02-14T17:52:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:54 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:54.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:54 smithi178 ceph-mon[30321]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:52:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:54 smithi184 ceph-mon[33536]: osdmap e463: 8 total, 8 up, 8 in 2024-02-14T17:52:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:54 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:54 smithi184 ceph-mon[33536]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:52:55.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:52:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[39811]: osdmap e464: 8 total, 8 up, 8 in 2024-02-14T17:52:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:55.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[30321]: osdmap e464: 8 total, 8 up, 8 in 2024-02-14T17:52:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:55.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:55 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:55 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:55 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:55 smithi184 ceph-mon[33536]: osdmap e464: 8 total, 8 up, 8 in 2024-02-14T17:52:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:52:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:55.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:55 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:52:56.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:56 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:52:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:56 smithi178 ceph-mon[39811]: osdmap e465: 8 total, 8 up, 8 in 2024-02-14T17:52:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:56 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:56 smithi178 ceph-mon[39811]: pgmap v703: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-02-14T17:52:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:56 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:52:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:56 smithi178 ceph-mon[30321]: osdmap e465: 8 total, 8 up, 8 in 2024-02-14T17:52:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:56 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:56 smithi178 ceph-mon[30321]: pgmap v703: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-02-14T17:52:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:56 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:52:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:56 smithi184 ceph-mon[33536]: osdmap e465: 8 total, 8 up, 8 in 2024-02-14T17:52:56.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:56.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:56 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:52:56.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:56 smithi184 ceph-mon[33536]: pgmap v703: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-02-14T17:52:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:57 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:52:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:57 smithi178 ceph-mon[30321]: osdmap e466: 8 total, 8 up, 8 in 2024-02-14T17:52:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:57 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:57.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:57 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:52:57.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:57 smithi178 ceph-mon[39811]: osdmap e466: 8 total, 8 up, 8 in 2024-02-14T17:52:57.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:57.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:57 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:57.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:57 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_tier","val": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:52:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:57 smithi184 ceph-mon[33536]: osdmap e466: 8 total, 8 up, 8 in 2024-02-14T17:52:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:57.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:57 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:52:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:52:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[30321]: osdmap e467: 8 total, 8 up, 8 in 2024-02-14T17:52:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[30321]: pgmap v706: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-14T17:52:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:58.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:52:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[39811]: osdmap e467: 8 total, 8 up, 8 in 2024-02-14T17:52:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[39811]: pgmap v706: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-14T17:52:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:58.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:58 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:58 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:52:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:58 smithi184 ceph-mon[33536]: osdmap e467: 8 total, 8 up, 8 in 2024-02-14T17:52:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:58 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:52:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:58 smithi184 ceph-mon[33536]: pgmap v706: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-14T17:52:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:58 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:58.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:58 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:52:59.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:52:59 smithi178 ceph-mon[30321]: osdmap e468: 8 total, 8 up, 8 in 2024-02-14T17:52:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:52:59 smithi178 ceph-mon[39811]: osdmap e468: 8 total, 8 up, 8 in 2024-02-14T17:52:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:52:59 smithi184 ceph-mon[33536]: osdmap e468: 8 total, 8 up, 8 in 2024-02-14T17:53:00.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:00 smithi178 ceph-mon[30321]: osdmap e469: 8 total, 8 up, 8 in 2024-02-14T17:53:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:00 smithi178 ceph-mon[30321]: pgmap v709: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:00 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:00 smithi178 ceph-mon[39811]: osdmap e469: 8 total, 8 up, 8 in 2024-02-14T17:53:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:00 smithi178 ceph-mon[39811]: pgmap v709: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:00 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:00 smithi184 ceph-mon[33536]: osdmap e469: 8 total, 8 up, 8 in 2024-02-14T17:53:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:00 smithi184 ceph-mon[33536]: pgmap v709: 228 pgs: 228 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:00 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:01.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:53:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:53:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:01 smithi184 ceph-mon[33536]: osdmap e470: 8 total, 8 up, 8 in 2024-02-14T17:53:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:01 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-85"}]: dispatch 2024-02-14T17:53:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:01 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-85"}]: dispatch 2024-02-14T17:53:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:01 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:01.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[30321]: osdmap e470: 8 total, 8 up, 8 in 2024-02-14T17:53:01.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:01.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-85"}]: dispatch 2024-02-14T17:53:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-85"}]: dispatch 2024-02-14T17:53:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[39811]: osdmap e470: 8 total, 8 up, 8 in 2024-02-14T17:53:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-85"}]: dispatch 2024-02-14T17:53:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-85"}]: dispatch 2024-02-14T17:53:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:01 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:02 smithi178 ceph-mon[39811]: osdmap e471: 8 total, 8 up, 8 in 2024-02-14T17:53:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:02 smithi178 ceph-mon[39811]: pgmap v712: 196 pgs: 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:53:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:02 smithi178 ceph-mon[30321]: osdmap e471: 8 total, 8 up, 8 in 2024-02-14T17:53:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:02 smithi178 ceph-mon[30321]: pgmap v712: 196 pgs: 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:53:02.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:02 smithi184 ceph-mon[33536]: osdmap e471: 8 total, 8 up, 8 in 2024-02-14T17:53:02.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:02 smithi184 ceph-mon[33536]: pgmap v712: 196 pgs: 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:53:03.561 INFO:tasks.workunit.client.0.smithi178.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (17850 ms) 2024-02-14T17:53:03.561 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-02-14T17:53:03.561 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22256 ms) 2024-02-14T17:53:03.561 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-02-14T17:53:03.561 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3011 ms) 2024-02-14T17:53:03.561 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-02-14T17:53:03.561 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3029 ms) 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3019 ms) 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3024 ms) 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4025 ms) 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38396 ms) 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.562 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16904 ms) 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4021 ms) 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3025 ms) 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24990 ms) 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14978 ms) 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.563 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5022 ms) 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4023 ms) 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: waiting for scrubs... 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: done waiting 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24972 ms) 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10062 ms) 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9046 ms) 2024-02-14T17:53:03.564 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-02-14T17:53:03.565 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:53:03.565 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9065 ms) 2024-02-14T17:53:03.565 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-02-14T17:53:03.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:03 smithi178 ceph-mon[39811]: osdmap e472: 8 total, 8 up, 8 in 2024-02-14T17:53:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:03 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:03 smithi178 ceph-mon[30321]: osdmap e472: 8 total, 8 up, 8 in 2024-02-14T17:53:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:03 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:03 smithi184 ceph-mon[33536]: osdmap e472: 8 total, 8 up, 8 in 2024-02-14T17:53:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:03.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:03 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:04 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:04 smithi178 ceph-mon[39811]: osdmap e473: 8 total, 8 up, 8 in 2024-02-14T17:53:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:04 smithi178 ceph-mon[39811]: pgmap v715: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:53:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:04 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:04 smithi178 ceph-mon[30321]: osdmap e473: 8 total, 8 up, 8 in 2024-02-14T17:53:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:04 smithi178 ceph-mon[30321]: pgmap v715: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:53:04.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:04 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:04.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:04 smithi184 ceph-mon[33536]: osdmap e473: 8 total, 8 up, 8 in 2024-02-14T17:53:04.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:04.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:04 smithi184 ceph-mon[33536]: pgmap v715: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:53:05.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:53:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:53:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:05.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:05 smithi178 ceph-mon[30321]: osdmap e474: 8 total, 8 up, 8 in 2024-02-14T17:53:05.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:05.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:05 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:05 smithi178 ceph-mon[39811]: osdmap e474: 8 total, 8 up, 8 in 2024-02-14T17:53:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:05.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:05 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:05.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:05 smithi184 ceph-mon[33536]: osdmap e474: 8 total, 8 up, 8 in 2024-02-14T17:53:05.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:05.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:05 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:06.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[30321]: osdmap e475: 8 total, 8 up, 8 in 2024-02-14T17:53:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_tier","val": "test-rados-api-smithi178-80903-89-test-flush"}]: dispatch 2024-02-14T17:53:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_tier","val": "test-rados-api-smithi178-80903-89-test-flush"}]: dispatch 2024-02-14T17:53:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[30321]: pgmap v718: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-14T17:53:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_tier","val": "test-rados-api-smithi178-80903-89-test-flush"}]': finished 2024-02-14T17:53:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[30321]: osdmap e476: 8 total, 8 up, 8 in 2024-02-14T17:53:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:53:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:53:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[39811]: osdmap e475: 8 total, 8 up, 8 in 2024-02-14T17:53:06.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_tier","val": "test-rados-api-smithi178-80903-89-test-flush"}]: dispatch 2024-02-14T17:53:06.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_tier","val": "test-rados-api-smithi178-80903-89-test-flush"}]: dispatch 2024-02-14T17:53:06.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[39811]: pgmap v718: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-14T17:53:06.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_tier","val": "test-rados-api-smithi178-80903-89-test-flush"}]': finished 2024-02-14T17:53:06.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[39811]: osdmap e476: 8 total, 8 up, 8 in 2024-02-14T17:53:06.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:53:06.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:53:06.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:06.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:06 smithi184 ceph-mon[33536]: osdmap e475: 8 total, 8 up, 8 in 2024-02-14T17:53:06.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_tier","val": "test-rados-api-smithi178-80903-89-test-flush"}]: dispatch 2024-02-14T17:53:06.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_tier","val": "test-rados-api-smithi178-80903-89-test-flush"}]: dispatch 2024-02-14T17:53:06.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:06 smithi184 ceph-mon[33536]: pgmap v718: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-14T17:53:06.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_tier","val": "test-rados-api-smithi178-80903-89-test-flush"}]': finished 2024-02-14T17:53:06.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:06 smithi184 ceph-mon[33536]: osdmap e476: 8 total, 8 up, 8 in 2024-02-14T17:53:06.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:53:06.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:53:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:07 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:07 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:53:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:07 smithi178 ceph-mon[39811]: osdmap e477: 8 total, 8 up, 8 in 2024-02-14T17:53:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:53:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:07 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:53:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:07 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:07 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:53:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:07 smithi178 ceph-mon[30321]: osdmap e477: 8 total, 8 up, 8 in 2024-02-14T17:53:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:53:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:07 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:53:07.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:07 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:07.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:07 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:53:07.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:07 smithi184 ceph-mon[33536]: osdmap e477: 8 total, 8 up, 8 in 2024-02-14T17:53:07.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:53:07.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:07 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:53:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:08 smithi178 ceph-mon[39811]: pgmap v721: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-14T17:53:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:08 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:53:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:08 smithi178 ceph-mon[39811]: osdmap e478: 8 total, 8 up, 8 in 2024-02-14T17:53:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:08 smithi178 ceph-mon[30321]: pgmap v721: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-14T17:53:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:08 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:53:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:08 smithi178 ceph-mon[30321]: osdmap e478: 8 total, 8 up, 8 in 2024-02-14T17:53:08.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:08 smithi184 ceph-mon[33536]: pgmap v721: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-14T17:53:08.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:08 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:53:08.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:08 smithi184 ceph-mon[33536]: osdmap e478: 8 total, 8 up, 8 in 2024-02-14T17:53:10.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:10 smithi184 ceph-mon[33536]: osdmap e479: 8 total, 8 up, 8 in 2024-02-14T17:53:10.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:10.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:10 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:10.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-87"}]: dispatch 2024-02-14T17:53:10.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:10 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-87"}]: dispatch 2024-02-14T17:53:10.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:10 smithi184 ceph-mon[33536]: pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:11.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[39811]: osdmap e479: 8 total, 8 up, 8 in 2024-02-14T17:53:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-87"}]: dispatch 2024-02-14T17:53:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-87"}]: dispatch 2024-02-14T17:53:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[39811]: pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[30321]: osdmap e479: 8 total, 8 up, 8 in 2024-02-14T17:53:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-87"}]: dispatch 2024-02-14T17:53:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-87"}]: dispatch 2024-02-14T17:53:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:10 smithi178 ceph-mon[30321]: pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:11.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:11 smithi184 ceph-mon[33536]: osdmap e480: 8 total, 8 up, 8 in 2024-02-14T17:53:11.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:11 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:11.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:53:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:53:12.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:11 smithi178 ceph-mon[39811]: osdmap e480: 8 total, 8 up, 8 in 2024-02-14T17:53:12.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:11 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:12.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:11 smithi178 ceph-mon[30321]: osdmap e480: 8 total, 8 up, 8 in 2024-02-14T17:53:12.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:11 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:12 smithi184 ceph-mon[33536]: osdmap e481: 8 total, 8 up, 8 in 2024-02-14T17:53:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:12 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:12 smithi184 ceph-mon[33536]: pgmap v727: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:53:13.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:12 smithi178 ceph-mon[30321]: osdmap e481: 8 total, 8 up, 8 in 2024-02-14T17:53:13.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:13.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:12 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:13.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:12 smithi178 ceph-mon[30321]: pgmap v727: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:53:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:12 smithi178 ceph-mon[39811]: osdmap e481: 8 total, 8 up, 8 in 2024-02-14T17:53:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:12 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:12 smithi178 ceph-mon[39811]: pgmap v727: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:53:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:13 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:13 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:13 smithi184 ceph-mon[33536]: osdmap e482: 8 total, 8 up, 8 in 2024-02-14T17:53:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:13 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:13.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[30321]: osdmap e482: 8 total, 8 up, 8 in 2024-02-14T17:53:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:14.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:14.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[39811]: osdmap e482: 8 total, 8 up, 8 in 2024-02-14T17:53:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:14.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:14 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:14 smithi184 ceph-mon[33536]: osdmap e483: 8 total, 8 up, 8 in 2024-02-14T17:53:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:14 smithi184 ceph-mon[33536]: pgmap v730: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:53:15.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:14 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:15.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:14 smithi178 ceph-mon[39811]: osdmap e483: 8 total, 8 up, 8 in 2024-02-14T17:53:15.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:14 smithi178 ceph-mon[39811]: pgmap v730: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:53:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:14 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:14 smithi178 ceph-mon[30321]: osdmap e483: 8 total, 8 up, 8 in 2024-02-14T17:53:15.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:14 smithi178 ceph-mon[30321]: pgmap v730: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:53:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:15 smithi184 ceph-mon[33536]: osdmap e484: 8 total, 8 up, 8 in 2024-02-14T17:53:15.983 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:15 smithi178 ceph-mon[39811]: osdmap e484: 8 total, 8 up, 8 in 2024-02-14T17:53:15.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:15 smithi178 ceph-mon[30321]: osdmap e484: 8 total, 8 up, 8 in 2024-02-14T17:53:15.984 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:53:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:53:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:16 smithi184 ceph-mon[33536]: osdmap e485: 8 total, 8 up, 8 in 2024-02-14T17:53:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-90"}]: dispatch 2024-02-14T17:53:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-90"}]: dispatch 2024-02-14T17:53:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:16 smithi184 ceph-mon[33536]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-02-14T17:53:17.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[39811]: osdmap e485: 8 total, 8 up, 8 in 2024-02-14T17:53:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-90"}]: dispatch 2024-02-14T17:53:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-90"}]: dispatch 2024-02-14T17:53:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[39811]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-02-14T17:53:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[30321]: osdmap e485: 8 total, 8 up, 8 in 2024-02-14T17:53:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-90"}]: dispatch 2024-02-14T17:53:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-90"}]: dispatch 2024-02-14T17:53:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:16 smithi178 ceph-mon[30321]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-02-14T17:53:17.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:17 smithi184 ceph-mon[33536]: osdmap e486: 8 total, 8 up, 8 in 2024-02-14T17:53:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:17 smithi178 ceph-mon[39811]: osdmap e486: 8 total, 8 up, 8 in 2024-02-14T17:53:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:17 smithi178 ceph-mon[30321]: osdmap e486: 8 total, 8 up, 8 in 2024-02-14T17:53:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:18 smithi184 ceph-mon[33536]: osdmap e487: 8 total, 8 up, 8 in 2024-02-14T17:53:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:18 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:18 smithi184 ceph-mon[33536]: pgmap v736: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:53:19.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:18 smithi178 ceph-mon[30321]: osdmap e487: 8 total, 8 up, 8 in 2024-02-14T17:53:19.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:19.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:18 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:19.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:18 smithi178 ceph-mon[30321]: pgmap v736: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:53:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:18 smithi178 ceph-mon[39811]: osdmap e487: 8 total, 8 up, 8 in 2024-02-14T17:53:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:18 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:19.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:18 smithi178 ceph-mon[39811]: pgmap v736: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:53:19.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:19 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:19.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:19 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:19.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:19 smithi184 ceph-mon[33536]: osdmap e488: 8 total, 8 up, 8 in 2024-02-14T17:53:19.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:20.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:19 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:19 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:19 smithi178 ceph-mon[30321]: osdmap e488: 8 total, 8 up, 8 in 2024-02-14T17:53:20.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:20.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:19 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:20.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:19 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:20.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:19 smithi178 ceph-mon[39811]: osdmap e488: 8 total, 8 up, 8 in 2024-02-14T17:53:20.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:20.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:20 smithi184 ceph-mon[33536]: osdmap e489: 8 total, 8 up, 8 in 2024-02-14T17:53:20.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:20 smithi184 ceph-mon[33536]: pgmap v739: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:21.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:20 smithi178 ceph-mon[30321]: osdmap e489: 8 total, 8 up, 8 in 2024-02-14T17:53:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:20 smithi178 ceph-mon[30321]: pgmap v739: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:20 smithi178 ceph-mon[39811]: osdmap e489: 8 total, 8 up, 8 in 2024-02-14T17:53:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:20 smithi178 ceph-mon[39811]: pgmap v739: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:21.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:53:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:53:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:21 smithi184 ceph-mon[33536]: osdmap e490: 8 total, 8 up, 8 in 2024-02-14T17:53:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:21 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-92"}]: dispatch 2024-02-14T17:53:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:21 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-92"}]: dispatch 2024-02-14T17:53:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:21 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:22.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[39811]: osdmap e490: 8 total, 8 up, 8 in 2024-02-14T17:53:22.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:22.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-92"}]: dispatch 2024-02-14T17:53:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-92"}]: dispatch 2024-02-14T17:53:22.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[30321]: osdmap e490: 8 total, 8 up, 8 in 2024-02-14T17:53:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-92"}]: dispatch 2024-02-14T17:53:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-92"}]: dispatch 2024-02-14T17:53:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:21 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:22 smithi184 ceph-mon[33536]: osdmap e491: 8 total, 8 up, 8 in 2024-02-14T17:53:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:22 smithi184 ceph-mon[33536]: pgmap v742: 196 pgs: 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:53:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:22 smithi184 ceph-mon[33536]: osdmap e492: 8 total, 8 up, 8 in 2024-02-14T17:53:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:22 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:23.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:22 smithi178 ceph-mon[30321]: osdmap e491: 8 total, 8 up, 8 in 2024-02-14T17:53:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:22 smithi178 ceph-mon[30321]: pgmap v742: 196 pgs: 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:53:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:22 smithi178 ceph-mon[30321]: osdmap e492: 8 total, 8 up, 8 in 2024-02-14T17:53:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:22 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:22 smithi178 ceph-mon[39811]: osdmap e491: 8 total, 8 up, 8 in 2024-02-14T17:53:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:22 smithi178 ceph-mon[39811]: pgmap v742: 196 pgs: 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:53:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:22 smithi178 ceph-mon[39811]: osdmap e492: 8 total, 8 up, 8 in 2024-02-14T17:53:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:22 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:24 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:24 smithi184 ceph-mon[33536]: osdmap e493: 8 total, 8 up, 8 in 2024-02-14T17:53:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:24 smithi184 ceph-mon[33536]: pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:53:25.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:24 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:24 smithi178 ceph-mon[39811]: osdmap e493: 8 total, 8 up, 8 in 2024-02-14T17:53:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:24 smithi178 ceph-mon[39811]: pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:53:25.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:24 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:25.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:24 smithi178 ceph-mon[30321]: osdmap e493: 8 total, 8 up, 8 in 2024-02-14T17:53:25.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:25.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:24 smithi178 ceph-mon[30321]: pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:53:25.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:25 smithi184 ceph-mon[33536]: osdmap e494: 8 total, 8 up, 8 in 2024-02-14T17:53:25.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:25 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:26.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:53:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:53:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:26.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:25 smithi178 ceph-mon[30321]: osdmap e494: 8 total, 8 up, 8 in 2024-02-14T17:53:26.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:25 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:25 smithi178 ceph-mon[39811]: osdmap e494: 8 total, 8 up, 8 in 2024-02-14T17:53:26.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:25 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:26.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:26 smithi184 ceph-mon[33536]: osdmap e495: 8 total, 8 up, 8 in 2024-02-14T17:53:26.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:26 smithi184 ceph-mon[33536]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:53:27.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:26 smithi178 ceph-mon[30321]: osdmap e495: 8 total, 8 up, 8 in 2024-02-14T17:53:27.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:26 smithi178 ceph-mon[30321]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:53:27.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:26 smithi178 ceph-mon[39811]: osdmap e495: 8 total, 8 up, 8 in 2024-02-14T17:53:27.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:26 smithi178 ceph-mon[39811]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:53:29.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:29 smithi184 ceph-mon[33536]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-02-14T17:53:29.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:29 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:29.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:29 smithi178 ceph-mon[30321]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-02-14T17:53:29.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:29 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:29 smithi178 ceph-mon[39811]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-02-14T17:53:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:29 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:31.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:31 smithi184 ceph-mon[33536]: pgmap v750: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:53:31.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:53:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:53:31.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:31 smithi178 ceph-mon[39811]: pgmap v750: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:53:31.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:31 smithi178 ceph-mon[30321]: pgmap v750: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:53:32.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:32 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:32.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:32 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:32 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:33.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:33 smithi184 ceph-mon[33536]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-14T17:53:33.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:33 smithi178 ceph-mon[39811]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-14T17:53:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:33 smithi178 ceph-mon[30321]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-14T17:53:35.687 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:35 smithi178 ceph-mon[30321]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 954 B/s wr, 4 op/s 2024-02-14T17:53:35.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:35 smithi178 ceph-mon[39811]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 954 B/s wr, 4 op/s 2024-02-14T17:53:35.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:35 smithi184 ceph-mon[33536]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 954 B/s wr, 4 op/s 2024-02-14T17:53:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:53:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:53:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:53:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi184 to 3248M 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: Adjusting osd_memory_target on smithi178 to 3248M 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:53:36.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:53:36.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:53:36.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:53:36.721 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:36 smithi184 ceph-mon[33536]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 96 B/s wr, 2 op/s 2024-02-14T17:53:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi184 to 3248M 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: Adjusting osd_memory_target on smithi178 to 3248M 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[39811]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 96 B/s wr, 2 op/s 2024-02-14T17:53:36.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:53:36.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi184 to 3248M 2024-02-14T17:53:36.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: Adjusting osd_memory_target on smithi178 to 3248M 2024-02-14T17:53:36.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:53:36.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:53:36.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:53:36.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:53:36.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:53:36.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:36 smithi178 ceph-mon[30321]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 96 B/s wr, 2 op/s 2024-02-14T17:53:37.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:37 smithi184 ceph-mon[33536]: osdmap e496: 8 total, 8 up, 8 in 2024-02-14T17:53:37.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:37 smithi178 ceph-mon[39811]: osdmap e496: 8 total, 8 up, 8 in 2024-02-14T17:53:37.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:37 smithi178 ceph-mon[30321]: osdmap e496: 8 total, 8 up, 8 in 2024-02-14T17:53:38.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:38 smithi184 ceph-mon[33536]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-14T17:53:38.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:38 smithi178 ceph-mon[39811]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-14T17:53:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:38 smithi178 ceph-mon[30321]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-14T17:53:41.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:41 smithi184 ceph-mon[33536]: pgmap v756: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-14T17:53:41.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:53:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:53:41.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:41 smithi178 ceph-mon[39811]: pgmap v756: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-14T17:53:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:41 smithi178 ceph-mon[30321]: pgmap v756: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-14T17:53:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:42 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:42 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-02-14T17:53:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:42 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-02-14T17:53:42.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:42 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:42 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-02-14T17:53:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:42 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-02-14T17:53:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:42 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:42 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-02-14T17:53:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:42 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]: dispatch 2024-02-14T17:53:43.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:43 smithi184 ceph-mon[33536]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:43.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:43 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]': finished 2024-02-14T17:53:43.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:43 smithi184 ceph-mon[33536]: osdmap e497: 8 total, 8 up, 8 in 2024-02-14T17:53:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:43 smithi178 ceph-mon[39811]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:43 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]': finished 2024-02-14T17:53:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:43 smithi178 ceph-mon[39811]: osdmap e497: 8 total, 8 up, 8 in 2024-02-14T17:53:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:43 smithi178 ceph-mon[30321]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:43 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "313.d", "id": [0, 1]}]': finished 2024-02-14T17:53:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:43 smithi178 ceph-mon[30321]: osdmap e497: 8 total, 8 up, 8 in 2024-02-14T17:53:44.719 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:53:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=cleanup t=2024-02-14T17:53:44.470211423Z level=info msg="Completed cleanup jobs" duration=2.533326ms 2024-02-14T17:53:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:44 smithi184 ceph-mon[33536]: osdmap e498: 8 total, 8 up, 8 in 2024-02-14T17:53:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:44 smithi184 ceph-mon[33536]: pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 903 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:44 smithi178 ceph-mon[39811]: osdmap e498: 8 total, 8 up, 8 in 2024-02-14T17:53:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:44 smithi178 ceph-mon[39811]: pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 903 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:44 smithi178 ceph-mon[30321]: osdmap e498: 8 total, 8 up, 8 in 2024-02-14T17:53:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:44 smithi178 ceph-mon[30321]: pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 903 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:46.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:53:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:53:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:46 smithi184 ceph-mon[33536]: osdmap e499: 8 total, 8 up, 8 in 2024-02-14T17:53:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:46 smithi184 ceph-mon[33536]: pgmap v762: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:53:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:46 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-94"}]: dispatch 2024-02-14T17:53:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:46 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-94"}]: dispatch 2024-02-14T17:53:46.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[39811]: osdmap e499: 8 total, 8 up, 8 in 2024-02-14T17:53:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[39811]: pgmap v762: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:53:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-94"}]: dispatch 2024-02-14T17:53:46.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-94"}]: dispatch 2024-02-14T17:53:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[30321]: osdmap e499: 8 total, 8 up, 8 in 2024-02-14T17:53:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[30321]: pgmap v762: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:53:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:53:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-94"}]: dispatch 2024-02-14T17:53:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:46 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-94"}]: dispatch 2024-02-14T17:53:47.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:47 smithi184 ceph-mon[33536]: osdmap e500: 8 total, 8 up, 8 in 2024-02-14T17:53:47.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:47 smithi178 ceph-mon[39811]: osdmap e500: 8 total, 8 up, 8 in 2024-02-14T17:53:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:47 smithi178 ceph-mon[30321]: osdmap e500: 8 total, 8 up, 8 in 2024-02-14T17:53:48.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:48 smithi178 ceph-mon[39811]: osdmap e501: 8 total, 8 up, 8 in 2024-02-14T17:53:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:48 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:48 smithi178 ceph-mon[39811]: pgmap v765: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:48 smithi178 ceph-mon[30321]: osdmap e501: 8 total, 8 up, 8 in 2024-02-14T17:53:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:48 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:48 smithi178 ceph-mon[30321]: pgmap v765: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:48 smithi184 ceph-mon[33536]: osdmap e501: 8 total, 8 up, 8 in 2024-02-14T17:53:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:48 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:48 smithi184 ceph-mon[33536]: pgmap v765: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:49.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:49 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:49 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:49 smithi178 ceph-mon[39811]: osdmap e502: 8 total, 8 up, 8 in 2024-02-14T17:53:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:49 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:49 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:49 smithi178 ceph-mon[30321]: osdmap e502: 8 total, 8 up, 8 in 2024-02-14T17:53:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:49.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:49 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:49.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:49 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:49.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:49 smithi184 ceph-mon[33536]: osdmap e502: 8 total, 8 up, 8 in 2024-02-14T17:53:49.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:50.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:50 smithi178 ceph-mon[39811]: osdmap e503: 8 total, 8 up, 8 in 2024-02-14T17:53:50.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:50 smithi178 ceph-mon[39811]: pgmap v768: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:50 smithi178 ceph-mon[30321]: osdmap e503: 8 total, 8 up, 8 in 2024-02-14T17:53:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:50 smithi178 ceph-mon[30321]: pgmap v768: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:50 smithi184 ceph-mon[33536]: osdmap e503: 8 total, 8 up, 8 in 2024-02-14T17:53:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:50 smithi184 ceph-mon[33536]: pgmap v768: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:51.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:53:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:53:53.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:53 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:53.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:53 smithi184 ceph-mon[33536]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 1.0 KiB/s wr, 2 op/s 2024-02-14T17:53:53.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:53 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:53.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:53 smithi178 ceph-mon[39811]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 1.0 KiB/s wr, 2 op/s 2024-02-14T17:53:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:53 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:53 smithi178 ceph-mon[30321]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 1.0 KiB/s wr, 2 op/s 2024-02-14T17:53:55.688 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:55 smithi178 ceph-mon[30321]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 747 B/s rd, 897 B/s wr, 2 op/s 2024-02-14T17:53:55.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:55 smithi178 ceph-mon[39811]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 747 B/s rd, 897 B/s wr, 2 op/s 2024-02-14T17:53:55.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:55 smithi184 ceph-mon[33536]: pgmap v770: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 747 B/s rd, 897 B/s wr, 2 op/s 2024-02-14T17:53:56.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:53:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:53:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:56.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:56 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:56.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:56 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:56 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:57 smithi184 ceph-mon[33536]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:53:57.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:57 smithi178 ceph-mon[39811]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:53:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:57 smithi178 ceph-mon[30321]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:53:59.719 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 17:53:59 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T17:53:59.505+0000 7fd827b51700 -1 snap_mapper.add_oid found existing snaps mapped on 102:8cef05f7:test-rados-api-smithi178-80903-97::foo:21, removing 2024-02-14T17:53:59.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:59 smithi184 ceph-mon[33536]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 624 B/s wr, 2 op/s 2024-02-14T17:53:59.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:53:59 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:59.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:59 smithi178 ceph-mon[39811]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 624 B/s wr, 2 op/s 2024-02-14T17:53:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:53:59 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:59 smithi178 ceph-mon[30321]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 624 B/s wr, 2 op/s 2024-02-14T17:53:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:53:59 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:59.833 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 17:53:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2[56763]: 2024-02-14T17:53:59.505+0000 7f0a35c1a700 -1 snap_mapper.add_oid found existing snaps mapped on 102:8cef05f7:test-rados-api-smithi178-80903-97::foo:21, removing 2024-02-14T17:53:59.834 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 17:53:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1[50608]: 2024-02-14T17:53:59.505+0000 7f33c0a88700 -1 snap_mapper.add_oid found existing snaps mapped on 102:8cef05f7:test-rados-api-smithi178-80903-97::foo:21, removing 2024-02-14T17:54:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:00 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-96"}]: dispatch 2024-02-14T17:54:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:00 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-96"}]: dispatch 2024-02-14T17:54:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:00 smithi184 ceph-mon[33536]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 944 B/s rd, 566 B/s wr, 2 op/s 2024-02-14T17:54:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:00 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-96"}]: dispatch 2024-02-14T17:54:00.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:00 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-96"}]: dispatch 2024-02-14T17:54:00.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:00 smithi178 ceph-mon[39811]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 944 B/s rd, 566 B/s wr, 2 op/s 2024-02-14T17:54:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:00 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-96"}]: dispatch 2024-02-14T17:54:00.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:00 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-96"}]: dispatch 2024-02-14T17:54:00.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:00 smithi178 ceph-mon[30321]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 944 B/s rd, 566 B/s wr, 2 op/s 2024-02-14T17:54:01.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:01 smithi184 ceph-mon[33536]: osdmap e504: 8 total, 8 up, 8 in 2024-02-14T17:54:01.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:54:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:54:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:01 smithi178 ceph-mon[39811]: osdmap e504: 8 total, 8 up, 8 in 2024-02-14T17:54:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:01 smithi178 ceph-mon[30321]: osdmap e504: 8 total, 8 up, 8 in 2024-02-14T17:54:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:02 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:02 smithi184 ceph-mon[33536]: osdmap e505: 8 total, 8 up, 8 in 2024-02-14T17:54:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:02 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:02.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:02 smithi184 ceph-mon[33536]: pgmap v776: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-14T17:54:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:02 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:02 smithi178 ceph-mon[39811]: osdmap e505: 8 total, 8 up, 8 in 2024-02-14T17:54:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:02 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:02 smithi178 ceph-mon[39811]: pgmap v776: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-14T17:54:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:02 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:02 smithi178 ceph-mon[30321]: osdmap e505: 8 total, 8 up, 8 in 2024-02-14T17:54:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:02 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:02.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:02 smithi178 ceph-mon[30321]: pgmap v776: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-14T17:54:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:03 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:03 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:03 smithi184 ceph-mon[33536]: osdmap e506: 8 total, 8 up, 8 in 2024-02-14T17:54:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:03 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:03 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:03 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:03 smithi178 ceph-mon[39811]: osdmap e506: 8 total, 8 up, 8 in 2024-02-14T17:54:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:03 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:03 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:03.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:03 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:03.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:03 smithi178 ceph-mon[30321]: osdmap e506: 8 total, 8 up, 8 in 2024-02-14T17:54:03.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:03.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:03 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:04.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:04 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:54:04.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:04 smithi184 ceph-mon[33536]: osdmap e507: 8 total, 8 up, 8 in 2024-02-14T17:54:04.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-98", "mode": "writeback"}]: dispatch 2024-02-14T17:54:04.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:04 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-98", "mode": "writeback"}]: dispatch 2024-02-14T17:54:04.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:04 smithi184 ceph-mon[33536]: pgmap v779: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-14T17:54:04.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:04 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:54:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:04 smithi178 ceph-mon[39811]: osdmap e507: 8 total, 8 up, 8 in 2024-02-14T17:54:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-98", "mode": "writeback"}]: dispatch 2024-02-14T17:54:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:04 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-98", "mode": "writeback"}]: dispatch 2024-02-14T17:54:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:04 smithi178 ceph-mon[39811]: pgmap v779: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-14T17:54:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:04 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:54:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:04 smithi178 ceph-mon[30321]: osdmap e507: 8 total, 8 up, 8 in 2024-02-14T17:54:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-98", "mode": "writeback"}]: dispatch 2024-02-14T17:54:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:04 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-98", "mode": "writeback"}]: dispatch 2024-02-14T17:54:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:04 smithi178 ceph-mon[30321]: pgmap v779: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-14T17:54:05.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:05 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:05.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:05 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-98", "mode": "writeback"}]': finished 2024-02-14T17:54:05.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:05 smithi178 ceph-mon[39811]: osdmap e508: 8 total, 8 up, 8 in 2024-02-14T17:54:05.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:05.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:05 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:05.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:05 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:05.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:05 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-98", "mode": "writeback"}]': finished 2024-02-14T17:54:05.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:05 smithi178 ceph-mon[30321]: osdmap e508: 8 total, 8 up, 8 in 2024-02-14T17:54:05.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:05.690 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:05 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:05.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:05 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:05 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-98", "mode": "writeback"}]': finished 2024-02-14T17:54:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:05 smithi184 ceph-mon[33536]: osdmap e508: 8 total, 8 up, 8 in 2024-02-14T17:54:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:05 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:06.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:54:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:54:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-98"}]': finished 2024-02-14T17:54:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:06 smithi184 ceph-mon[33536]: osdmap e509: 8 total, 8 up, 8 in 2024-02-14T17:54:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:54:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:06 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:54:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:06 smithi184 ceph-mon[33536]: pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:54:06.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-98"}]': finished 2024-02-14T17:54:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:06 smithi178 ceph-mon[30321]: osdmap e509: 8 total, 8 up, 8 in 2024-02-14T17:54:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:54:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:06 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:54:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:06 smithi178 ceph-mon[30321]: pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:54:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-98"}]': finished 2024-02-14T17:54:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:06 smithi178 ceph-mon[39811]: osdmap e509: 8 total, 8 up, 8 in 2024-02-14T17:54:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:54:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:06 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:54:06.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:06 smithi178 ceph-mon[39811]: pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:54:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:07 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:07 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:54:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:07 smithi184 ceph-mon[33536]: osdmap e510: 8 total, 8 up, 8 in 2024-02-14T17:54:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:54:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:07 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:54:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:07 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:07 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:54:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:07 smithi178 ceph-mon[30321]: osdmap e510: 8 total, 8 up, 8 in 2024-02-14T17:54:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:54:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:07 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:54:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:07 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:07 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:54:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:07 smithi178 ceph-mon[39811]: osdmap e510: 8 total, 8 up, 8 in 2024-02-14T17:54:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:54:07.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:07 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:54:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:08 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_count","val": "1"}]': finished 2024-02-14T17:54:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:08 smithi184 ceph-mon[33536]: osdmap e511: 8 total, 8 up, 8 in 2024-02-14T17:54:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:08 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:08 smithi184 ceph-mon[33536]: pgmap v785: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:54:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:08 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:54:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_count","val": "1"}]': finished 2024-02-14T17:54:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[39811]: osdmap e511: 8 total, 8 up, 8 in 2024-02-14T17:54:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[39811]: pgmap v785: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:54:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:54:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_count","val": "1"}]': finished 2024-02-14T17:54:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[30321]: osdmap e511: 8 total, 8 up, 8 in 2024-02-14T17:54:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[30321]: pgmap v785: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:54:08.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:08 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:54:09.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:09 smithi184 ceph-mon[33536]: osdmap e512: 8 total, 8 up, 8 in 2024-02-14T17:54:09.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:54:09.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:09 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:54:09.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:09 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "target_max_objects","val": "250"}]': finished 2024-02-14T17:54:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:09 smithi178 ceph-mon[39811]: osdmap e512: 8 total, 8 up, 8 in 2024-02-14T17:54:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:54:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:09 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:54:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:09 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "target_max_objects","val": "250"}]': finished 2024-02-14T17:54:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:09 smithi178 ceph-mon[30321]: osdmap e512: 8 total, 8 up, 8 in 2024-02-14T17:54:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:54:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:09 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:54:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:09 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-98","var": "target_max_objects","val": "250"}]': finished 2024-02-14T17:54:10.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:10 smithi184 ceph-mon[33536]: osdmap e513: 8 total, 8 up, 8 in 2024-02-14T17:54:10.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:10.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:10 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:10.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:10 smithi184 ceph-mon[33536]: pgmap v788: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:10.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:10 smithi178 ceph-mon[39811]: osdmap e513: 8 total, 8 up, 8 in 2024-02-14T17:54:10.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:10.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:10 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:10 smithi178 ceph-mon[39811]: pgmap v788: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:10 smithi178 ceph-mon[30321]: osdmap e513: 8 total, 8 up, 8 in 2024-02-14T17:54:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:10 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:10 smithi178 ceph-mon[30321]: pgmap v788: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:11.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:54:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:54:11.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:11 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:54:11.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:11 smithi184 ceph-mon[33536]: osdmap e514: 8 total, 8 up, 8 in 2024-02-14T17:54:11.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:11.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:11 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:11.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:11 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:11.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:11 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:54:11.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:11 smithi178 ceph-mon[39811]: osdmap e514: 8 total, 8 up, 8 in 2024-02-14T17:54:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:11 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:11.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:11 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:11 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:54:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:11 smithi178 ceph-mon[30321]: osdmap e514: 8 total, 8 up, 8 in 2024-02-14T17:54:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:11 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98"}]: dispatch 2024-02-14T17:54:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:11 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:12.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:12 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:12.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:12 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98"}]': finished 2024-02-14T17:54:12.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:12 smithi184 ceph-mon[33536]: osdmap e515: 8 total, 8 up, 8 in 2024-02-14T17:54:12.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:12 smithi184 ceph-mon[33536]: pgmap v791: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:12.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:12 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:12.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:12 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98"}]': finished 2024-02-14T17:54:12.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:12 smithi178 ceph-mon[39811]: osdmap e515: 8 total, 8 up, 8 in 2024-02-14T17:54:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:12 smithi178 ceph-mon[39811]: pgmap v791: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:12 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:12 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-98"}]': finished 2024-02-14T17:54:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:12 smithi178 ceph-mon[30321]: osdmap e515: 8 total, 8 up, 8 in 2024-02-14T17:54:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:12 smithi178 ceph-mon[30321]: pgmap v791: 228 pgs: 228 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:13.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:13 smithi184 ceph-mon[33536]: osdmap e516: 8 total, 8 up, 8 in 2024-02-14T17:54:13.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:13 smithi178 ceph-mon[39811]: osdmap e516: 8 total, 8 up, 8 in 2024-02-14T17:54:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:13 smithi178 ceph-mon[30321]: osdmap e516: 8 total, 8 up, 8 in 2024-02-14T17:54:14.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[39811]: osdmap e517: 8 total, 8 up, 8 in 2024-02-14T17:54:14.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[39811]: pgmap v794: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[39811]: osdmap e518: 8 total, 8 up, 8 in 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[30321]: osdmap e517: 8 total, 8 up, 8 in 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[30321]: pgmap v794: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:14 smithi178 ceph-mon[30321]: osdmap e518: 8 total, 8 up, 8 in 2024-02-14T17:54:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:14 smithi184 ceph-mon[33536]: osdmap e517: 8 total, 8 up, 8 in 2024-02-14T17:54:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:14 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:14 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:14 smithi184 ceph-mon[33536]: pgmap v794: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:14 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:14 smithi184 ceph-mon[33536]: osdmap e518: 8 total, 8 up, 8 in 2024-02-14T17:54:15.734 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:54:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:15.734 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:15.734 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:15.734 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:54:15.734 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[30321]: osdmap e519: 8 total, 8 up, 8 in 2024-02-14T17:54:15.734 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:15.734 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:15.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:15.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:15.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:54:15.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[39811]: osdmap e519: 8 total, 8 up, 8 in 2024-02-14T17:54:15.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:15.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:15 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:54:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:54:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:15 smithi184 ceph-mon[33536]: osdmap e519: 8 total, 8 up, 8 in 2024-02-14T17:54:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:15 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:16.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:16.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[39811]: pgmap v797: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:16.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-100"}]': finished 2024-02-14T17:54:16.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[39811]: osdmap e520: 8 total, 8 up, 8 in 2024-02-14T17:54:16.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-100", "mode": "writeback"}]: dispatch 2024-02-14T17:54:16.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-100", "mode": "writeback"}]: dispatch 2024-02-14T17:54:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[30321]: pgmap v797: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-100"}]': finished 2024-02-14T17:54:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[30321]: osdmap e520: 8 total, 8 up, 8 in 2024-02-14T17:54:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-100", "mode": "writeback"}]: dispatch 2024-02-14T17:54:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:16 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-100", "mode": "writeback"}]: dispatch 2024-02-14T17:54:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:16 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:16 smithi184 ceph-mon[33536]: pgmap v797: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-6", "overlaypool": "test-rados-api-smithi178-80903-100"}]': finished 2024-02-14T17:54:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:16 smithi184 ceph-mon[33536]: osdmap e520: 8 total, 8 up, 8 in 2024-02-14T17:54:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-100", "mode": "writeback"}]: dispatch 2024-02-14T17:54:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:16 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-100", "mode": "writeback"}]: dispatch 2024-02-14T17:54:17.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:17 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:17 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-100", "mode": "writeback"}]': finished 2024-02-14T17:54:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:17 smithi178 ceph-mon[39811]: osdmap e521: 8 total, 8 up, 8 in 2024-02-14T17:54:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:17.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:17 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:17.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:17 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:17.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:17 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-100", "mode": "writeback"}]': finished 2024-02-14T17:54:17.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:17 smithi178 ceph-mon[30321]: osdmap e521: 8 total, 8 up, 8 in 2024-02-14T17:54:17.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:17.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:17 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:17.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:17 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:17.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:17 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-100", "mode": "writeback"}]': finished 2024-02-14T17:54:17.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:17 smithi184 ceph-mon[33536]: osdmap e521: 8 total, 8 up, 8 in 2024-02-14T17:54:17.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:17.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:17 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:18.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:18 smithi178 ceph-mon[39811]: pgmap v800: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:18 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:54:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:18 smithi178 ceph-mon[39811]: osdmap e522: 8 total, 8 up, 8 in 2024-02-14T17:54:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:18.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:18 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:18 smithi178 ceph-mon[30321]: pgmap v800: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:18 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:54:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:18 smithi178 ceph-mon[30321]: osdmap e522: 8 total, 8 up, 8 in 2024-02-14T17:54:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:18 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:18.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:18 smithi184 ceph-mon[33536]: pgmap v800: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:18 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:54:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:18 smithi184 ceph-mon[33536]: osdmap e522: 8 total, 8 up, 8 in 2024-02-14T17:54:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:18 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:20.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:20 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:54:20.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:20 smithi184 ceph-mon[33536]: osdmap e523: 8 total, 8 up, 8 in 2024-02-14T17:54:20.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:20.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:20 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:20.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:20 smithi184 ceph-mon[33536]: pgmap v803: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:20.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:20 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:54:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:20 smithi178 ceph-mon[30321]: osdmap e523: 8 total, 8 up, 8 in 2024-02-14T17:54:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:20 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:20 smithi178 ceph-mon[30321]: pgmap v803: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:20.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:20 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:54:20.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:20 smithi178 ceph-mon[39811]: osdmap e523: 8 total, 8 up, 8 in 2024-02-14T17:54:20.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:20.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:20 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:20.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:20 smithi178 ceph-mon[39811]: pgmap v803: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:21.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:54:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:54:21.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:21 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:21.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:21 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:54:21.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:21 smithi184 ceph-mon[33536]: osdmap e524: 8 total, 8 up, 8 in 2024-02-14T17:54:21.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:54:21.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:21 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:54:21.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:21 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:21.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:21.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:54:21.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[30321]: osdmap e524: 8 total, 8 up, 8 in 2024-02-14T17:54:21.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:54:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:54:21.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:21.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:21.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:54:21.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[39811]: osdmap e524: 8 total, 8 up, 8 in 2024-02-14T17:54:21.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:54:21.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:54:21.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:21 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:22.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-14T17:54:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[30321]: osdmap e525: 8 total, 8 up, 8 in 2024-02-14T17:54:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[30321]: pgmap v806: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-14T17:54:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[39811]: osdmap e525: 8 total, 8 up, 8 in 2024-02-14T17:54:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:22.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:22.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:22 smithi178 ceph-mon[39811]: pgmap v806: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:22 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:22 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-14T17:54:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:22 smithi184 ceph-mon[33536]: osdmap e525: 8 total, 8 up, 8 in 2024-02-14T17:54:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:22 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:22 smithi184 ceph-mon[33536]: pgmap v806: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:23.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:23 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:54:23.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:23 smithi178 ceph-mon[39811]: osdmap e526: 8 total, 8 up, 8 in 2024-02-14T17:54:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:23.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:23 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:23.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:23 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:54:23.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:23 smithi178 ceph-mon[30321]: osdmap e526: 8 total, 8 up, 8 in 2024-02-14T17:54:23.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:23.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:23 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:23.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:23 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]': finished 2024-02-14T17:54:23.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:23 smithi184 ceph-mon[33536]: osdmap e526: 8 total, 8 up, 8 in 2024-02-14T17:54:23.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:23.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:23 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100"}]: dispatch 2024-02-14T17:54:24.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:24 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100"}]': finished 2024-02-14T17:54:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:24 smithi178 ceph-mon[39811]: osdmap e527: 8 total, 8 up, 8 in 2024-02-14T17:54:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:24 smithi178 ceph-mon[39811]: pgmap v809: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:24 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100"}]': finished 2024-02-14T17:54:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:24 smithi178 ceph-mon[30321]: osdmap e527: 8 total, 8 up, 8 in 2024-02-14T17:54:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:24 smithi178 ceph-mon[30321]: pgmap v809: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:24.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:24 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-100"}]': finished 2024-02-14T17:54:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:24 smithi184 ceph-mon[33536]: osdmap e527: 8 total, 8 up, 8 in 2024-02-14T17:54:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:24 smithi184 ceph-mon[33536]: pgmap v809: 228 pgs: 228 active+clean; 8.4 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:25.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:25 smithi178 ceph-mon[39811]: osdmap e528: 8 total, 8 up, 8 in 2024-02-14T17:54:25.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:25 smithi178 ceph-mon[30321]: osdmap e528: 8 total, 8 up, 8 in 2024-02-14T17:54:25.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:54:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:54:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:25.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:25 smithi184 ceph-mon[33536]: osdmap e528: 8 total, 8 up, 8 in 2024-02-14T17:54:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:26 smithi178 ceph-mon[30321]: osdmap e529: 8 total, 8 up, 8 in 2024-02-14T17:54:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:26 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:26 smithi178 ceph-mon[30321]: pgmap v812: 228 pgs: 7 creating+peering, 25 unknown, 196 active+clean; 8.4 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:26.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:26 smithi178 ceph-mon[39811]: osdmap e529: 8 total, 8 up, 8 in 2024-02-14T17:54:26.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:26.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:26 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:26.835 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:26 smithi178 ceph-mon[39811]: pgmap v812: 228 pgs: 7 creating+peering, 25 unknown, 196 active+clean; 8.4 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:26.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:26 smithi184 ceph-mon[33536]: osdmap e529: 8 total, 8 up, 8 in 2024-02-14T17:54:26.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:26.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:26 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:26.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:26 smithi184 ceph-mon[33536]: pgmap v812: 228 pgs: 7 creating+peering, 25 unknown, 196 active+clean; 8.4 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:27.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[30321]: osdmap e530: 8 total, 8 up, 8 in 2024-02-14T17:54:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:27.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:27.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:27.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[39811]: osdmap e530: 8 total, 8 up, 8 in 2024-02-14T17:54:27.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:27.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:27.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:27 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:27.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:27 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:27.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:27 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:27.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:27 smithi184 ceph-mon[33536]: osdmap e530: 8 total, 8 up, 8 in 2024-02-14T17:54:27.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:27.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:27.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:27 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:28.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:54:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[39811]: osdmap e531: 8 total, 8 up, 8 in 2024-02-14T17:54:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:54:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:54:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[39811]: pgmap v815: 228 pgs: 7 creating+peering, 25 unknown, 196 active+clean; 8.4 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:54:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[39811]: osdmap e532: 8 total, 8 up, 8 in 2024-02-14T17:54:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:54:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:54:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:54:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[30321]: osdmap e531: 8 total, 8 up, 8 in 2024-02-14T17:54:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:54:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:54:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[30321]: pgmap v815: 228 pgs: 7 creating+peering, 25 unknown, 196 active+clean; 8.4 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:54:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[30321]: osdmap e532: 8 total, 8 up, 8 in 2024-02-14T17:54:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:54:28.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:28 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:54:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:28 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:54:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:28 smithi184 ceph-mon[33536]: osdmap e531: 8 total, 8 up, 8 in 2024-02-14T17:54:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:54:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:28 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:54:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:28 smithi184 ceph-mon[33536]: pgmap v815: 228 pgs: 7 creating+peering, 25 unknown, 196 active+clean; 8.4 MiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:28 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:28 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:54:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:28 smithi184 ceph-mon[33536]: osdmap e532: 8 total, 8 up, 8 in 2024-02-14T17:54:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:54:28.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:28 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:54:30.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:54:30.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[39811]: osdmap e533: 8 total, 8 up, 8 in 2024-02-14T17:54:30.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:30.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:30.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-102"}]: dispatch 2024-02-14T17:54:30.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[39811]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-102"}]: dispatch 2024-02-14T17:54:30.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[39811]: pgmap v818: 228 pgs: 7 creating+peering, 25 unknown, 196 active+clean; 8.4 MiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:30.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:54:30.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[30321]: osdmap e533: 8 total, 8 up, 8 in 2024-02-14T17:54:30.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:30.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:30.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-102"}]: dispatch 2024-02-14T17:54:30.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[30321]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-102"}]: dispatch 2024-02-14T17:54:30.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:30 smithi178 ceph-mon[30321]: pgmap v818: 228 pgs: 7 creating+peering, 25 unknown, 196 active+clean; 8.4 MiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:30 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:54:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:30 smithi184 ceph-mon[33536]: osdmap e533: 8 total, 8 up, 8 in 2024-02-14T17:54:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:30 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-6"}]: dispatch 2024-02-14T17:54:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3437997140' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-102"}]: dispatch 2024-02-14T17:54:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:30 smithi184 ceph-mon[33536]: from='client.35477 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-6", "tierpool": "test-rados-api-smithi178-80903-102"}]: dispatch 2024-02-14T17:54:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:30 smithi184 ceph-mon[33536]: pgmap v818: 228 pgs: 7 creating+peering, 25 unknown, 196 active+clean; 8.4 MiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:31.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:31 smithi184 ceph-mon[33536]: osdmap e534: 8 total, 8 up, 8 in 2024-02-14T17:54:31.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:54:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:54:31.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:31 smithi178 ceph-mon[39811]: osdmap e534: 8 total, 8 up, 8 in 2024-02-14T17:54:31.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:31 smithi178 ceph-mon[30321]: osdmap e534: 8 total, 8 up, 8 in 2024-02-14T17:54:32.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[39811]: osdmap e535: 8 total, 8 up, 8 in 2024-02-14T17:54:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[39811]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[39811]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[39811]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[39811]: pgmap v821: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[30321]: osdmap e535: 8 total, 8 up, 8 in 2024-02-14T17:54:32.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[30321]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[30321]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:32.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[30321]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:32.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:32 smithi178 ceph-mon[30321]: pgmap v821: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:32 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:32 smithi184 ceph-mon[33536]: osdmap e535: 8 total, 8 up, 8 in 2024-02-14T17:54:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:32 smithi184 ceph-mon[33536]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:32 smithi184 ceph-mon[33536]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:32 smithi184 ceph-mon[33536]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:32 smithi184 ceph-mon[33536]: pgmap v821: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:34.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:33 smithi178 ceph-mon[39811]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:54:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:33 smithi178 ceph-mon[39811]: osdmap e536: 8 total, 8 up, 8 in 2024-02-14T17:54:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi178-80903-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:34.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:33 smithi178 ceph-mon[39811]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi178-80903-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:33 smithi178 ceph-mon[30321]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:54:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:33 smithi178 ceph-mon[30321]: osdmap e536: 8 total, 8 up, 8 in 2024-02-14T17:54:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi178-80903-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:34.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:33 smithi178 ceph-mon[30321]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi178-80903-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:33 smithi184 ceph-mon[33536]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:54:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:33 smithi184 ceph-mon[33536]: osdmap e536: 8 total, 8 up, 8 in 2024-02-14T17:54:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi178-80903-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:34.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:33 smithi184 ceph-mon[33536]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi178-80903-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:54:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:34 smithi178 ceph-mon[30321]: osdmap e537: 8 total, 8 up, 8 in 2024-02-14T17:54:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:34 smithi178 ceph-mon[30321]: pgmap v824: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:34 smithi178 ceph-mon[30321]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi178-80903-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi178-80903-104"}]': finished 2024-02-14T17:54:35.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:34 smithi178 ceph-mon[30321]: osdmap e538: 8 total, 8 up, 8 in 2024-02-14T17:54:35.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:34 smithi178 ceph-mon[39811]: osdmap e537: 8 total, 8 up, 8 in 2024-02-14T17:54:35.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:34 smithi178 ceph-mon[39811]: pgmap v824: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:35.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:34 smithi178 ceph-mon[39811]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi178-80903-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi178-80903-104"}]': finished 2024-02-14T17:54:35.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:34 smithi178 ceph-mon[39811]: osdmap e538: 8 total, 8 up, 8 in 2024-02-14T17:54:35.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:34 smithi184 ceph-mon[33536]: osdmap e537: 8 total, 8 up, 8 in 2024-02-14T17:54:35.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:34 smithi184 ceph-mon[33536]: pgmap v824: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:35.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:34 smithi184 ceph-mon[33536]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi178-80903-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi178-80903-104"}]': finished 2024-02-14T17:54:35.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:34 smithi184 ceph-mon[33536]: osdmap e538: 8 total, 8 up, 8 in 2024-02-14T17:54:36.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:54:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:54:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:36.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:36 smithi178 ceph-mon[39811]: osdmap e539: 8 total, 8 up, 8 in 2024-02-14T17:54:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:36 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:36 smithi178 ceph-mon[39811]: pgmap v827: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:36 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:54:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:36 smithi178 ceph-mon[30321]: osdmap e539: 8 total, 8 up, 8 in 2024-02-14T17:54:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:36 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:36 smithi178 ceph-mon[30321]: pgmap v827: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:36 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:54:36.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:36 smithi184 ceph-mon[33536]: osdmap e539: 8 total, 8 up, 8 in 2024-02-14T17:54:36.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:36 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:36.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:36 smithi184 ceph-mon[33536]: pgmap v827: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:36.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:36 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:54:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[39811]: osdmap e540: 8 total, 8 up, 8 in 2024-02-14T17:54:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:54:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:54:37.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:54:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[30321]: osdmap e540: 8 total, 8 up, 8 in 2024-02-14T17:54:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:54:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:54:37.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:37 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:54:37.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:37 smithi184 ceph-mon[33536]: osdmap e540: 8 total, 8 up, 8 in 2024-02-14T17:54:37.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:37.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:37 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:37.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:37 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:54:37.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:37 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:54:37.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:37 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:54:38.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:38 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:38 smithi178 ceph-mon[39811]: osdmap e541: 8 total, 8 up, 8 in 2024-02-14T17:54:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:38 smithi178 ceph-mon[39811]: pgmap v830: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:38 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:38 smithi178 ceph-mon[30321]: osdmap e541: 8 total, 8 up, 8 in 2024-02-14T17:54:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:38 smithi178 ceph-mon[30321]: pgmap v830: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:38.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:38 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:38.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:38 smithi184 ceph-mon[33536]: osdmap e541: 8 total, 8 up, 8 in 2024-02-14T17:54:38.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:38 smithi184 ceph-mon[33536]: pgmap v830: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:39.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:39 smithi178 ceph-mon[39811]: osdmap e542: 8 total, 8 up, 8 in 2024-02-14T17:54:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:39 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:39 smithi178 ceph-mon[30321]: osdmap e542: 8 total, 8 up, 8 in 2024-02-14T17:54:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:39 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:39.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:39 smithi184 ceph-mon[33536]: osdmap e542: 8 total, 8 up, 8 in 2024-02-14T17:54:39.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:39.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:39 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[39811]: osdmap e543: 8 total, 8 up, 8 in 2024-02-14T17:54:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[39811]: pgmap v833: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]': finished 2024-02-14T17:54:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[39811]: osdmap e544: 8 total, 8 up, 8 in 2024-02-14T17:54:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-107", "overlaypool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:40.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-107", "overlaypool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[30321]: osdmap e543: 8 total, 8 up, 8 in 2024-02-14T17:54:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[30321]: pgmap v833: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]': finished 2024-02-14T17:54:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[30321]: osdmap e544: 8 total, 8 up, 8 in 2024-02-14T17:54:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-107", "overlaypool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:40.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:40 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-107", "overlaypool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:40 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:40 smithi184 ceph-mon[33536]: osdmap e543: 8 total, 8 up, 8 in 2024-02-14T17:54:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:40 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:40 smithi184 ceph-mon[33536]: pgmap v833: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:40 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]': finished 2024-02-14T17:54:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:40 smithi184 ceph-mon[33536]: osdmap e544: 8 total, 8 up, 8 in 2024-02-14T17:54:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-107", "overlaypool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:40 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-107", "overlaypool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:41.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:54:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:54:41.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:41 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:41.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:41 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-107", "overlaypool": "test-rados-api-smithi178-80903-107-cache"}]': finished 2024-02-14T17:54:41.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:41 smithi184 ceph-mon[33536]: osdmap e545: 8 total, 8 up, 8 in 2024-02-14T17:54:41.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-107-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:41.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:41 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-107-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:41.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:41 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:41 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-107", "overlaypool": "test-rados-api-smithi178-80903-107-cache"}]': finished 2024-02-14T17:54:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:41 smithi178 ceph-mon[39811]: osdmap e545: 8 total, 8 up, 8 in 2024-02-14T17:54:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-107-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:41 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-107-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:41 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:41 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-107", "overlaypool": "test-rados-api-smithi178-80903-107-cache"}]': finished 2024-02-14T17:54:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:41 smithi178 ceph-mon[30321]: osdmap e545: 8 total, 8 up, 8 in 2024-02-14T17:54:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-107-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:41 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-107-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:42 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:42 smithi178 ceph-mon[39811]: pgmap v836: 236 pgs: 236 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:42 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:42 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-107-cache", "mode": "writeback"}]': finished 2024-02-14T17:54:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:42 smithi178 ceph-mon[39811]: osdmap e546: 8 total, 8 up, 8 in 2024-02-14T17:54:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:42 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:42 smithi178 ceph-mon[30321]: pgmap v836: 236 pgs: 236 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:42 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:42 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-107-cache", "mode": "writeback"}]': finished 2024-02-14T17:54:42.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:42 smithi178 ceph-mon[30321]: osdmap e546: 8 total, 8 up, 8 in 2024-02-14T17:54:42.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:42 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:42.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:42 smithi184 ceph-mon[33536]: pgmap v836: 236 pgs: 236 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:42.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:42 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:42.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:42 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-107-cache", "mode": "writeback"}]': finished 2024-02-14T17:54:42.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:42 smithi184 ceph-mon[33536]: osdmap e546: 8 total, 8 up, 8 in 2024-02-14T17:54:43.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-107"}]: dispatch 2024-02-14T17:54:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:43 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-107"}]: dispatch 2024-02-14T17:54:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:43 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:54:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-107"}]: dispatch 2024-02-14T17:54:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:43 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-107"}]: dispatch 2024-02-14T17:54:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:43 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:54:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:43.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-107"}]: dispatch 2024-02-14T17:54:43.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:43 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-107"}]: dispatch 2024-02-14T17:54:43.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:43 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:54:43.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:44 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-107"}]': finished 2024-02-14T17:54:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:44 smithi184 ceph-mon[33536]: osdmap e547: 8 total, 8 up, 8 in 2024-02-14T17:54:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:44 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:44 smithi184 ceph-mon[33536]: pgmap v839: 236 pgs: 236 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:44 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-107"}]': finished 2024-02-14T17:54:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:44 smithi178 ceph-mon[30321]: osdmap e547: 8 total, 8 up, 8 in 2024-02-14T17:54:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:44 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:44 smithi178 ceph-mon[30321]: pgmap v839: 236 pgs: 236 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:44 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-107"}]': finished 2024-02-14T17:54:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:44 smithi178 ceph-mon[39811]: osdmap e547: 8 total, 8 up, 8 in 2024-02-14T17:54:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3018948009' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:45.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:44 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]: dispatch 2024-02-14T17:54:45.085 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:44 smithi178 ceph-mon[39811]: pgmap v839: 236 pgs: 236 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:54:45.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:45 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:45.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:45 smithi184 ceph-mon[33536]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]': finished 2024-02-14T17:54:45.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:45 smithi184 ceph-mon[33536]: osdmap e548: 8 total, 8 up, 8 in 2024-02-14T17:54:45.984 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:54:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:54:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:45.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:45 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:45.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:45 smithi178 ceph-mon[39811]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]': finished 2024-02-14T17:54:45.984 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:45 smithi178 ceph-mon[39811]: osdmap e548: 8 total, 8 up, 8 in 2024-02-14T17:54:45.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:45 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:45.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:45 smithi178 ceph-mon[30321]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-107", "tierpool": "test-rados-api-smithi178-80903-107-cache"}]': finished 2024-02-14T17:54:45.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:45 smithi178 ceph-mon[30321]: osdmap e548: 8 total, 8 up, 8 in 2024-02-14T17:54:46.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:46 smithi184 ceph-mon[33536]: osdmap e549: 8 total, 8 up, 8 in 2024-02-14T17:54:46.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:46 smithi184 ceph-mon[33536]: pgmap v842: 204 pgs: 204 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:54:47.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:46 smithi178 ceph-mon[39811]: osdmap e549: 8 total, 8 up, 8 in 2024-02-14T17:54:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:46 smithi178 ceph-mon[39811]: pgmap v842: 204 pgs: 204 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:54:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:46 smithi178 ceph-mon[30321]: osdmap e549: 8 total, 8 up, 8 in 2024-02-14T17:54:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:46 smithi178 ceph-mon[30321]: pgmap v842: 204 pgs: 204 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:54:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:47 smithi184 ceph-mon[33536]: osdmap e550: 8 total, 8 up, 8 in 2024-02-14T17:54:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:47 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:47 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:47.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:47 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:48.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[39811]: osdmap e550: 8 total, 8 up, 8 in 2024-02-14T17:54:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:48.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:48.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:48.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[30321]: osdmap e550: 8 total, 8 up, 8 in 2024-02-14T17:54:48.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:48.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:48.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:48.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:48.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:48.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:47 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:54:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:48 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:54:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:48 smithi184 ceph-mon[33536]: osdmap e551: 8 total, 8 up, 8 in 2024-02-14T17:54:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:48 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:48 smithi184 ceph-mon[33536]: pgmap v845: 172 pgs: 172 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:48 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:54:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:48 smithi178 ceph-mon[39811]: osdmap e551: 8 total, 8 up, 8 in 2024-02-14T17:54:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:48 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:48 smithi178 ceph-mon[39811]: pgmap v845: 172 pgs: 172 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:49.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:48 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:54:49.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:48 smithi178 ceph-mon[30321]: osdmap e551: 8 total, 8 up, 8 in 2024-02-14T17:54:49.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:49.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:48 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:49.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:48 smithi178 ceph-mon[30321]: pgmap v845: 172 pgs: 172 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:49.972 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:49 smithi184 ceph-mon[33536]: osdmap e552: 8 total, 8 up, 8 in 2024-02-14T17:54:50.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:49 smithi178 ceph-mon[39811]: osdmap e552: 8 total, 8 up, 8 in 2024-02-14T17:54:50.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:49 smithi178 ceph-mon[30321]: osdmap e552: 8 total, 8 up, 8 in 2024-02-14T17:54:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:50 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:54:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:50 smithi184 ceph-mon[33536]: osdmap e553: 8 total, 8 up, 8 in 2024-02-14T17:54:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:50 smithi184 ceph-mon[33536]: pgmap v848: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:51.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:50 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:54:51.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:50 smithi178 ceph-mon[39811]: osdmap e553: 8 total, 8 up, 8 in 2024-02-14T17:54:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:50 smithi178 ceph-mon[39811]: pgmap v848: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:50 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:54:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:50 smithi178 ceph-mon[30321]: osdmap e553: 8 total, 8 up, 8 in 2024-02-14T17:54:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:50 smithi178 ceph-mon[30321]: pgmap v848: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 572 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:51.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:51 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:51.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:51 smithi184 ceph-mon[33536]: osdmap e554: 8 total, 8 up, 8 in 2024-02-14T17:54:51.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:51.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:51 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:51.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:54:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:54:52.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:51 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:51 smithi178 ceph-mon[39811]: osdmap e554: 8 total, 8 up, 8 in 2024-02-14T17:54:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:51 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:51 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:51 smithi178 ceph-mon[30321]: osdmap e554: 8 total, 8 up, 8 in 2024-02-14T17:54:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:51 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:52 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:52 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:52 smithi184 ceph-mon[33536]: osdmap e555: 8 total, 8 up, 8 in 2024-02-14T17:54:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:52 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:52 smithi184 ceph-mon[33536]: pgmap v851: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[39811]: osdmap e555: 8 total, 8 up, 8 in 2024-02-14T17:54:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[39811]: pgmap v851: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[30321]: osdmap e555: 8 total, 8 up, 8 in 2024-02-14T17:54:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:52 smithi178 ceph-mon[30321]: pgmap v851: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:53 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]': finished 2024-02-14T17:54:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:53 smithi184 ceph-mon[33536]: osdmap e556: 8 total, 8 up, 8 in 2024-02-14T17:54:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:53 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-109", "overlaypool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:53 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-109", "overlaypool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:54.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:53 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]': finished 2024-02-14T17:54:54.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:53 smithi178 ceph-mon[39811]: osdmap e556: 8 total, 8 up, 8 in 2024-02-14T17:54:54.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:53 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-109", "overlaypool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:54.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:53 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-109", "overlaypool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:53 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]': finished 2024-02-14T17:54:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:53 smithi178 ceph-mon[30321]: osdmap e556: 8 total, 8 up, 8 in 2024-02-14T17:54:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:53 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-109", "overlaypool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:53 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-109", "overlaypool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:54:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:54 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-109", "overlaypool": "test-rados-api-smithi178-80903-109-cache"}]': finished 2024-02-14T17:54:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:54 smithi184 ceph-mon[33536]: osdmap e557: 8 total, 8 up, 8 in 2024-02-14T17:54:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:54 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:54 smithi184 ceph-mon[33536]: pgmap v854: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:55.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:54 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-109", "overlaypool": "test-rados-api-smithi178-80903-109-cache"}]': finished 2024-02-14T17:54:55.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:54 smithi178 ceph-mon[39811]: osdmap e557: 8 total, 8 up, 8 in 2024-02-14T17:54:55.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:55.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:54 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:55.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:54 smithi178 ceph-mon[39811]: pgmap v854: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:55.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:54 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-109", "overlaypool": "test-rados-api-smithi178-80903-109-cache"}]': finished 2024-02-14T17:54:55.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:54 smithi178 ceph-mon[30321]: osdmap e557: 8 total, 8 up, 8 in 2024-02-14T17:54:55.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:55.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:54 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:54:55.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:54 smithi178 ceph-mon[30321]: pgmap v854: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:55 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:55 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-109-cache", "mode": "writeback"}]': finished 2024-02-14T17:54:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:55 smithi184 ceph-mon[33536]: osdmap e558: 8 total, 8 up, 8 in 2024-02-14T17:54:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:55 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:56.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:54:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:54:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:56.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:55 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:56.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:55 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-109-cache", "mode": "writeback"}]': finished 2024-02-14T17:54:56.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:55 smithi178 ceph-mon[30321]: osdmap e558: 8 total, 8 up, 8 in 2024-02-14T17:54:56.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:56.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:55 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:55 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:54:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:55 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-109-cache", "mode": "writeback"}]': finished 2024-02-14T17:54:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:55 smithi178 ceph-mon[39811]: osdmap e558: 8 total, 8 up, 8 in 2024-02-14T17:54:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:56.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:55 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:54:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:56 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:54:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:56 smithi184 ceph-mon[33536]: osdmap e559: 8 total, 8 up, 8 in 2024-02-14T17:54:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:56 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:56 smithi184 ceph-mon[33536]: pgmap v857: 212 pgs: 212 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:56 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:54:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:56 smithi184 ceph-mon[33536]: osdmap e560: 8 total, 8 up, 8 in 2024-02-14T17:54:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:56.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:56 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:57.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:54:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[39811]: osdmap e559: 8 total, 8 up, 8 in 2024-02-14T17:54:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[39811]: pgmap v857: 212 pgs: 212 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:54:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[39811]: osdmap e560: 8 total, 8 up, 8 in 2024-02-14T17:54:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:54:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[30321]: osdmap e559: 8 total, 8 up, 8 in 2024-02-14T17:54:57.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:57.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:54:57.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[30321]: pgmap v857: 212 pgs: 212 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:57.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:54:57.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[30321]: osdmap e560: 8 total, 8 up, 8 in 2024-02-14T17:54:57.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:57.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:56 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:54:57.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:57 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:57.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:57 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:54:57.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:57 smithi184 ceph-mon[33536]: osdmap e561: 8 total, 8 up, 8 in 2024-02-14T17:54:57.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:54:57.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:57 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:54:58.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:57 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:57 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:54:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:57 smithi178 ceph-mon[39811]: osdmap e561: 8 total, 8 up, 8 in 2024-02-14T17:54:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:54:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:57 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:54:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:57 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:54:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:57 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:54:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:57 smithi178 ceph-mon[30321]: osdmap e561: 8 total, 8 up, 8 in 2024-02-14T17:54:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:54:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:57 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:54:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:58 smithi184 ceph-mon[33536]: pgmap v860: 212 pgs: 212 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:58 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:54:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:58 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:58 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-14T17:54:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:58 smithi184 ceph-mon[33536]: osdmap e562: 8 total, 8 up, 8 in 2024-02-14T17:54:59.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:58 smithi178 ceph-mon[39811]: pgmap v860: 212 pgs: 212 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:58 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:54:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:58 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:58 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-14T17:54:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:58 smithi178 ceph-mon[39811]: osdmap e562: 8 total, 8 up, 8 in 2024-02-14T17:54:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:58 smithi178 ceph-mon[30321]: pgmap v860: 212 pgs: 212 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:58 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:54:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:58 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:58 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-14T17:54:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:58 smithi178 ceph-mon[30321]: osdmap e562: 8 total, 8 up, 8 in 2024-02-14T17:54:59.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:54:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:54:59 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:00.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:54:59 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:54:59 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:00 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:55:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:00 smithi184 ceph-mon[33536]: osdmap e563: 8 total, 8 up, 8 in 2024-02-14T17:55:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:55:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:00 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:55:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:00 smithi184 ceph-mon[33536]: pgmap v863: 212 pgs: 212 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:01.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:00 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:55:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:00 smithi178 ceph-mon[39811]: osdmap e563: 8 total, 8 up, 8 in 2024-02-14T17:55:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:55:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:00 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:55:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:00 smithi178 ceph-mon[39811]: pgmap v863: 212 pgs: 212 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:00 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:55:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:00 smithi178 ceph-mon[30321]: osdmap e563: 8 total, 8 up, 8 in 2024-02-14T17:55:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:55:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:00 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]: dispatch 2024-02-14T17:55:01.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:00 smithi178 ceph-mon[30321]: pgmap v863: 212 pgs: 212 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:01.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:55:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:55:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:01 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]': finished 2024-02-14T17:55:01.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:01 smithi184 ceph-mon[33536]: osdmap e564: 8 total, 8 up, 8 in 2024-02-14T17:55:02.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:01 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]': finished 2024-02-14T17:55:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:01 smithi178 ceph-mon[39811]: osdmap e564: 8 total, 8 up, 8 in 2024-02-14T17:55:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:01 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-109", "tierpool": "test-rados-api-smithi178-80903-109-cache"}]': finished 2024-02-14T17:55:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:01 smithi178 ceph-mon[30321]: osdmap e564: 8 total, 8 up, 8 in 2024-02-14T17:55:02.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:02 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:02.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:02 smithi184 ceph-mon[33536]: osdmap e565: 8 total, 8 up, 8 in 2024-02-14T17:55:02.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:02 smithi184 ceph-mon[33536]: pgmap v866: 180 pgs: 180 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:03.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:02 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:03.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:02 smithi178 ceph-mon[30321]: osdmap e565: 8 total, 8 up, 8 in 2024-02-14T17:55:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:02 smithi178 ceph-mon[30321]: pgmap v866: 180 pgs: 180 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:02 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:02 smithi178 ceph-mon[39811]: osdmap e565: 8 total, 8 up, 8 in 2024-02-14T17:55:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:02 smithi178 ceph-mon[39811]: pgmap v866: 180 pgs: 180 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:03 smithi184 ceph-mon[33536]: osdmap e566: 8 total, 8 up, 8 in 2024-02-14T17:55:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:03 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:04.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:03 smithi178 ceph-mon[39811]: osdmap e566: 8 total, 8 up, 8 in 2024-02-14T17:55:04.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:04.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:03 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:03 smithi178 ceph-mon[30321]: osdmap e566: 8 total, 8 up, 8 in 2024-02-14T17:55:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:03 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:04.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:04 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:55:04.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:04 smithi184 ceph-mon[33536]: osdmap e567: 8 total, 8 up, 8 in 2024-02-14T17:55:04.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:04.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:04 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:04.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:04 smithi184 ceph-mon[33536]: pgmap v869: 172 pgs: 172 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:05.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:04 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:55:05.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:04 smithi178 ceph-mon[39811]: osdmap e567: 8 total, 8 up, 8 in 2024-02-14T17:55:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:04 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:04 smithi178 ceph-mon[39811]: pgmap v869: 172 pgs: 172 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:04 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:55:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:04 smithi178 ceph-mon[30321]: osdmap e567: 8 total, 8 up, 8 in 2024-02-14T17:55:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3616696318' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:04 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]: dispatch 2024-02-14T17:55:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:04 smithi178 ceph-mon[30321]: pgmap v869: 172 pgs: 172 active+clean; 582 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:05.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:05 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:05.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:05 smithi184 ceph-mon[33536]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:55:05.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:05 smithi184 ceph-mon[33536]: osdmap e568: 8 total, 8 up, 8 in 2024-02-14T17:55:06.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:55:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:55:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:06.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:05 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:06.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:05 smithi178 ceph-mon[30321]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:55:06.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:05 smithi178 ceph-mon[30321]: osdmap e568: 8 total, 8 up, 8 in 2024-02-14T17:55:06.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:05 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:05 smithi178 ceph-mon[39811]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-109"}]': finished 2024-02-14T17:55:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:05 smithi178 ceph-mon[39811]: osdmap e568: 8 total, 8 up, 8 in 2024-02-14T17:55:06.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:06 smithi184 ceph-mon[33536]: osdmap e569: 8 total, 8 up, 8 in 2024-02-14T17:55:06.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:06.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:06 smithi184 ceph-mon[33536]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:06.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:06 smithi184 ceph-mon[33536]: pgmap v872: 164 pgs: 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-14T17:55:07.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:06 smithi178 ceph-mon[39811]: osdmap e569: 8 total, 8 up, 8 in 2024-02-14T17:55:07.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:07.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:06 smithi178 ceph-mon[39811]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:07.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:06 smithi178 ceph-mon[39811]: pgmap v872: 164 pgs: 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-14T17:55:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:06 smithi178 ceph-mon[30321]: osdmap e569: 8 total, 8 up, 8 in 2024-02-14T17:55:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:06 smithi178 ceph-mon[30321]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:06 smithi178 ceph-mon[30321]: pgmap v872: 164 pgs: 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-14T17:55:08.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:07 smithi178 ceph-mon[30321]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]': finished 2024-02-14T17:55:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:07 smithi178 ceph-mon[30321]: osdmap e570: 8 total, 8 up, 8 in 2024-02-14T17:55:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:07 smithi178 ceph-mon[30321]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:07 smithi178 ceph-mon[39811]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]': finished 2024-02-14T17:55:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:07 smithi178 ceph-mon[39811]: osdmap e570: 8 total, 8 up, 8 in 2024-02-14T17:55:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:07 smithi178 ceph-mon[39811]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:08.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:07 smithi184 ceph-mon[33536]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi178-80903-104"}]': finished 2024-02-14T17:55:08.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:07 smithi184 ceph-mon[33536]: osdmap e570: 8 total, 8 up, 8 in 2024-02-14T17:55:08.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1600286527' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:08.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:07 smithi184 ceph-mon[33536]: from='client.56945 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]: dispatch 2024-02-14T17:55:09.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[30321]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]': finished 2024-02-14T17:55:09.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[30321]: osdmap e571: 8 total, 8 up, 8 in 2024-02-14T17:55:09.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:09.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:09.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:55:09.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[30321]: pgmap v875: 164 pgs: 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-14T17:55:09.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[39811]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]': finished 2024-02-14T17:55:09.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[39811]: osdmap e571: 8 total, 8 up, 8 in 2024-02-14T17:55:09.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:09.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:09.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:55:09.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:08 smithi178 ceph-mon[39811]: pgmap v875: 164 pgs: 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-14T17:55:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:08 smithi184 ceph-mon[33536]: from='client.56945 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi178-80903-104"}]': finished 2024-02-14T17:55:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:08 smithi184 ceph-mon[33536]: osdmap e571: 8 total, 8 up, 8 in 2024-02-14T17:55:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:55:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:08 smithi184 ceph-mon[33536]: pgmap v875: 164 pgs: 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-14T17:55:10.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:55:10.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:09 smithi178 ceph-mon[39811]: osdmap e572: 8 total, 8 up, 8 in 2024-02-14T17:55:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:10.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:55:10.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:09 smithi178 ceph-mon[30321]: osdmap e572: 8 total, 8 up, 8 in 2024-02-14T17:55:10.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:10.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi178-80903-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:55:10.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:09 smithi184 ceph-mon[33536]: osdmap e572: 8 total, 8 up, 8 in 2024-02-14T17:55:10.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:11.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:10 smithi178 ceph-mon[39811]: osdmap e573: 8 total, 8 up, 8 in 2024-02-14T17:55:11.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:10 smithi178 ceph-mon[39811]: pgmap v878: 164 pgs: 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:11.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:11.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:10 smithi178 ceph-mon[39811]: osdmap e574: 8 total, 8 up, 8 in 2024-02-14T17:55:11.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:10 smithi178 ceph-mon[30321]: osdmap e573: 8 total, 8 up, 8 in 2024-02-14T17:55:11.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:10 smithi178 ceph-mon[30321]: pgmap v878: 164 pgs: 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:11.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:11.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:10 smithi178 ceph-mon[30321]: osdmap e574: 8 total, 8 up, 8 in 2024-02-14T17:55:11.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:10 smithi184 ceph-mon[33536]: osdmap e573: 8 total, 8 up, 8 in 2024-02-14T17:55:11.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:10 smithi184 ceph-mon[33536]: pgmap v878: 164 pgs: 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:11.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi178-80903-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:11.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:10 smithi184 ceph-mon[33536]: osdmap e574: 8 total, 8 up, 8 in 2024-02-14T17:55:11.718 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:55:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:55:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:11 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:11 smithi178 ceph-mon[39811]: osdmap e575: 8 total, 8 up, 8 in 2024-02-14T17:55:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:12.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:11 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:12.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:11 smithi178 ceph-mon[30321]: osdmap e575: 8 total, 8 up, 8 in 2024-02-14T17:55:12.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:12.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:11 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:12.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:11 smithi184 ceph-mon[33536]: osdmap e575: 8 total, 8 up, 8 in 2024-02-14T17:55:12.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:13.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:12 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:13.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:12 smithi178 ceph-mon[39811]: pgmap v881: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:13.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:13.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:12 smithi178 ceph-mon[39811]: osdmap e576: 8 total, 8 up, 8 in 2024-02-14T17:55:13.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:12 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:12 smithi178 ceph-mon[30321]: pgmap v881: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:12 smithi178 ceph-mon[30321]: osdmap e576: 8 total, 8 up, 8 in 2024-02-14T17:55:13.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:12 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:12 smithi184 ceph-mon[33536]: pgmap v881: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:12 smithi184 ceph-mon[33536]: osdmap e576: 8 total, 8 up, 8 in 2024-02-14T17:55:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:13 smithi178 ceph-mon[39811]: osdmap e577: 8 total, 8 up, 8 in 2024-02-14T17:55:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-112"}]: dispatch 2024-02-14T17:55:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:14.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:14.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:13 smithi178 ceph-mon[30321]: osdmap e577: 8 total, 8 up, 8 in 2024-02-14T17:55:14.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-112"}]: dispatch 2024-02-14T17:55:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:13 smithi184 ceph-mon[33536]: osdmap e577: 8 total, 8 up, 8 in 2024-02-14T17:55:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-112"}]: dispatch 2024-02-14T17:55:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:14 smithi178 ceph-mon[39811]: pgmap v884: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-112"}]': finished 2024-02-14T17:55:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:14 smithi178 ceph-mon[39811]: osdmap e578: 8 total, 8 up, 8 in 2024-02-14T17:55:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:14 smithi178 ceph-mon[30321]: pgmap v884: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-112"}]': finished 2024-02-14T17:55:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:14 smithi178 ceph-mon[30321]: osdmap e578: 8 total, 8 up, 8 in 2024-02-14T17:55:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:15.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:14 smithi184 ceph-mon[33536]: pgmap v884: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-112"}]': finished 2024-02-14T17:55:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:14 smithi184 ceph-mon[33536]: osdmap e578: 8 total, 8 up, 8 in 2024-02-14T17:55:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:16.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:55:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:55:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:17.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:16 smithi178 ceph-mon[39811]: osdmap e579: 8 total, 8 up, 8 in 2024-02-14T17:55:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112"}]: dispatch 2024-02-14T17:55:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:16 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:16 smithi178 ceph-mon[39811]: pgmap v887: 204 pgs: 204 active+clean; 582 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:55:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:16 smithi178 ceph-mon[30321]: osdmap e579: 8 total, 8 up, 8 in 2024-02-14T17:55:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112"}]: dispatch 2024-02-14T17:55:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:16 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:16 smithi178 ceph-mon[30321]: pgmap v887: 204 pgs: 204 active+clean; 582 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:55:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:16 smithi184 ceph-mon[33536]: osdmap e579: 8 total, 8 up, 8 in 2024-02-14T17:55:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112"}]: dispatch 2024-02-14T17:55:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:16 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:17.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:16 smithi184 ceph-mon[33536]: pgmap v887: 204 pgs: 204 active+clean; 582 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:55:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112"}]': finished 2024-02-14T17:55:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:17 smithi178 ceph-mon[39811]: osdmap e580: 8 total, 8 up, 8 in 2024-02-14T17:55:18.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112"}]': finished 2024-02-14T17:55:18.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:17 smithi178 ceph-mon[30321]: osdmap e580: 8 total, 8 up, 8 in 2024-02-14T17:55:18.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-112"}]': finished 2024-02-14T17:55:18.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:17 smithi184 ceph-mon[33536]: osdmap e580: 8 total, 8 up, 8 in 2024-02-14T17:55:19.002 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:18 smithi178 ceph-mon[39811]: osdmap e581: 8 total, 8 up, 8 in 2024-02-14T17:55:19.002 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:18 smithi178 ceph-mon[39811]: pgmap v890: 172 pgs: 172 active+clean; 582 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:19.002 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:19 smithi178 ceph-mon[39811]: osdmap e582: 8 total, 8 up, 8 in 2024-02-14T17:55:19.002 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:19.002 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:18 smithi178 ceph-mon[30321]: osdmap e581: 8 total, 8 up, 8 in 2024-02-14T17:55:19.002 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:18 smithi178 ceph-mon[30321]: pgmap v890: 172 pgs: 172 active+clean; 582 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:19.002 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:18 smithi178 ceph-mon[30321]: osdmap e582: 8 total, 8 up, 8 in 2024-02-14T17:55:19.003 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:19 smithi184 ceph-mon[33536]: osdmap e581: 8 total, 8 up, 8 in 2024-02-14T17:55:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:19 smithi184 ceph-mon[33536]: pgmap v890: 172 pgs: 172 active+clean; 582 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:19 smithi184 ceph-mon[33536]: osdmap e582: 8 total, 8 up, 8 in 2024-02-14T17:55:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:21.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:20 smithi178 ceph-mon[39811]: osdmap e583: 8 total, 8 up, 8 in 2024-02-14T17:55:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:21.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:20 smithi178 ceph-mon[39811]: pgmap v893: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:20 smithi178 ceph-mon[30321]: osdmap e583: 8 total, 8 up, 8 in 2024-02-14T17:55:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:21.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:20 smithi178 ceph-mon[30321]: pgmap v893: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:21.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:21.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:20 smithi184 ceph-mon[33536]: osdmap e583: 8 total, 8 up, 8 in 2024-02-14T17:55:21.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:21.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:20 smithi184 ceph-mon[33536]: pgmap v893: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:21.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:55:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:55:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:21 smithi178 ceph-mon[39811]: osdmap e584: 8 total, 8 up, 8 in 2024-02-14T17:55:22.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-114"}]: dispatch 2024-02-14T17:55:22.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:22.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:21 smithi178 ceph-mon[30321]: osdmap e584: 8 total, 8 up, 8 in 2024-02-14T17:55:22.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-114"}]: dispatch 2024-02-14T17:55:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:21 smithi184 ceph-mon[33536]: osdmap e584: 8 total, 8 up, 8 in 2024-02-14T17:55:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-114"}]: dispatch 2024-02-14T17:55:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-114"}]': finished 2024-02-14T17:55:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[39811]: osdmap e585: 8 total, 8 up, 8 in 2024-02-14T17:55:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-114", "mode": "writeback"}]: dispatch 2024-02-14T17:55:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[39811]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:55:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-114", "mode": "writeback"}]': finished 2024-02-14T17:55:23.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[39811]: osdmap e586: 8 total, 8 up, 8 in 2024-02-14T17:55:23.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-114"}]': finished 2024-02-14T17:55:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[30321]: osdmap e585: 8 total, 8 up, 8 in 2024-02-14T17:55:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-114", "mode": "writeback"}]: dispatch 2024-02-14T17:55:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[30321]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:55:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-114", "mode": "writeback"}]': finished 2024-02-14T17:55:23.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:22 smithi178 ceph-mon[30321]: osdmap e586: 8 total, 8 up, 8 in 2024-02-14T17:55:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:22 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-114"}]': finished 2024-02-14T17:55:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:22 smithi184 ceph-mon[33536]: osdmap e585: 8 total, 8 up, 8 in 2024-02-14T17:55:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-114", "mode": "writeback"}]: dispatch 2024-02-14T17:55:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:22 smithi184 ceph-mon[33536]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:22 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:55:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-114", "mode": "writeback"}]': finished 2024-02-14T17:55:23.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:22 smithi184 ceph-mon[33536]: osdmap e586: 8 total, 8 up, 8 in 2024-02-14T17:55:24.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:24.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:24.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:25.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:24 smithi178 ceph-mon[39811]: osdmap e587: 8 total, 8 up, 8 in 2024-02-14T17:55:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114"}]: dispatch 2024-02-14T17:55:25.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:24 smithi178 ceph-mon[39811]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:25.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:25.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:24 smithi178 ceph-mon[30321]: osdmap e587: 8 total, 8 up, 8 in 2024-02-14T17:55:25.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114"}]: dispatch 2024-02-14T17:55:25.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:24 smithi178 ceph-mon[30321]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:25.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:25.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:24 smithi184 ceph-mon[33536]: osdmap e587: 8 total, 8 up, 8 in 2024-02-14T17:55:25.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114"}]: dispatch 2024-02-14T17:55:25.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:24 smithi184 ceph-mon[33536]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:55:26.034 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:55:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:55:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:26.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:26 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:55:26.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114"}]': finished 2024-02-14T17:55:26.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:26 smithi178 ceph-mon[39811]: osdmap e588: 8 total, 8 up, 8 in 2024-02-14T17:55:26.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:26 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:55:26.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114"}]': finished 2024-02-14T17:55:26.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:26 smithi178 ceph-mon[30321]: osdmap e588: 8 total, 8 up, 8 in 2024-02-14T17:55:26.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:26 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:55:26.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-114"}]': finished 2024-02-14T17:55:26.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:26 smithi184 ceph-mon[33536]: osdmap e588: 8 total, 8 up, 8 in 2024-02-14T17:55:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:27 smithi178 ceph-mon[30321]: osdmap e589: 8 total, 8 up, 8 in 2024-02-14T17:55:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:27 smithi178 ceph-mon[30321]: pgmap v902: 172 pgs: 172 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:27 smithi178 ceph-mon[30321]: osdmap e590: 8 total, 8 up, 8 in 2024-02-14T17:55:27.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:27.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:27 smithi178 ceph-mon[39811]: osdmap e589: 8 total, 8 up, 8 in 2024-02-14T17:55:27.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:27 smithi178 ceph-mon[39811]: pgmap v902: 172 pgs: 172 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:27.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:27 smithi178 ceph-mon[39811]: osdmap e590: 8 total, 8 up, 8 in 2024-02-14T17:55:27.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:27 smithi184 ceph-mon[33536]: osdmap e589: 8 total, 8 up, 8 in 2024-02-14T17:55:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:27 smithi184 ceph-mon[33536]: pgmap v902: 172 pgs: 172 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:27 smithi184 ceph-mon[33536]: osdmap e590: 8 total, 8 up, 8 in 2024-02-14T17:55:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:29.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:28 smithi178 ceph-mon[30321]: osdmap e591: 8 total, 8 up, 8 in 2024-02-14T17:55:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:28 smithi178 ceph-mon[30321]: pgmap v905: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:29.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:28 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:28 smithi178 ceph-mon[39811]: osdmap e591: 8 total, 8 up, 8 in 2024-02-14T17:55:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:28 smithi178 ceph-mon[39811]: pgmap v905: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:29.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:28 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:29.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:29.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:28 smithi184 ceph-mon[33536]: osdmap e591: 8 total, 8 up, 8 in 2024-02-14T17:55:29.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:28 smithi184 ceph-mon[33536]: pgmap v905: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:29.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:28 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:30.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:29 smithi178 ceph-mon[39811]: osdmap e592: 8 total, 8 up, 8 in 2024-02-14T17:55:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:29 smithi178 ceph-mon[30321]: osdmap e592: 8 total, 8 up, 8 in 2024-02-14T17:55:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:30.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:29 smithi184 ceph-mon[33536]: osdmap e592: 8 total, 8 up, 8 in 2024-02-14T17:55:30.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:30 smithi178 ceph-mon[30321]: osdmap e593: 8 total, 8 up, 8 in 2024-02-14T17:55:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-116"}]: dispatch 2024-02-14T17:55:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:30 smithi178 ceph-mon[30321]: pgmap v908: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:30 smithi178 ceph-mon[39811]: osdmap e593: 8 total, 8 up, 8 in 2024-02-14T17:55:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-116"}]: dispatch 2024-02-14T17:55:31.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:30 smithi178 ceph-mon[39811]: pgmap v908: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:31.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:31.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:30 smithi184 ceph-mon[33536]: osdmap e593: 8 total, 8 up, 8 in 2024-02-14T17:55:31.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-116"}]: dispatch 2024-02-14T17:55:31.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:30 smithi184 ceph-mon[33536]: pgmap v908: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:31.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:55:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:55:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-116"}]': finished 2024-02-14T17:55:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:31 smithi178 ceph-mon[39811]: osdmap e594: 8 total, 8 up, 8 in 2024-02-14T17:55:32.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-116", "mode": "writeback"}]: dispatch 2024-02-14T17:55:32.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-116"}]': finished 2024-02-14T17:55:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:31 smithi178 ceph-mon[30321]: osdmap e594: 8 total, 8 up, 8 in 2024-02-14T17:55:32.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-116", "mode": "writeback"}]: dispatch 2024-02-14T17:55:32.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-116"}]': finished 2024-02-14T17:55:32.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:31 smithi184 ceph-mon[33536]: osdmap e594: 8 total, 8 up, 8 in 2024-02-14T17:55:32.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-116", "mode": "writeback"}]: dispatch 2024-02-14T17:55:33.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:55:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-116", "mode": "writeback"}]': finished 2024-02-14T17:55:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[39811]: osdmap e595: 8 total, 8 up, 8 in 2024-02-14T17:55:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-14T17:55:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[39811]: from='mon.0 v1:172.21.15.178:0/1654782959' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-14T17:55:33.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[39811]: pgmap v911: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:55:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:55:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-116", "mode": "writeback"}]': finished 2024-02-14T17:55:33.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[30321]: osdmap e595: 8 total, 8 up, 8 in 2024-02-14T17:55:33.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-14T17:55:33.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[30321]: from='mon.0 v1:172.21.15.178:0/1654782959' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-14T17:55:33.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:32 smithi178 ceph-mon[30321]: pgmap v911: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:55:33.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:32 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:33.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:32 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:55:33.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-116", "mode": "writeback"}]': finished 2024-02-14T17:55:33.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:32 smithi184 ceph-mon[33536]: osdmap e595: 8 total, 8 up, 8 in 2024-02-14T17:55:33.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-14T17:55:33.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:32 smithi184 ceph-mon[33536]: from='mon.0 v1:172.21.15.178:0/1654782959' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-14T17:55:33.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:32 smithi184 ceph-mon[33536]: pgmap v911: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:55:34.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:33 smithi178 ceph-mon[39811]: 326.7 scrub starts 2024-02-14T17:55:34.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:33 smithi178 ceph-mon[39811]: 326.7 scrub ok 2024-02-14T17:55:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:33 smithi178 ceph-mon[30321]: 326.7 scrub starts 2024-02-14T17:55:34.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:33 smithi178 ceph-mon[30321]: 326.7 scrub ok 2024-02-14T17:55:34.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:33 smithi184 ceph-mon[33536]: 326.7 scrub starts 2024-02-14T17:55:34.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:33 smithi184 ceph-mon[33536]: 326.7 scrub ok 2024-02-14T17:55:35.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:34 smithi178 ceph-mon[39811]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 922 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-02-14T17:55:35.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:34 smithi178 ceph-mon[30321]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 922 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-02-14T17:55:35.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:34 smithi184 ceph-mon[33536]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 922 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-02-14T17:55:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:55:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:55:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:36 smithi184 ceph-mon[33536]: pgmap v913: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-14T17:55:36.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:36 smithi178 ceph-mon[39811]: pgmap v913: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-14T17:55:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:36 smithi178 ceph-mon[30321]: pgmap v913: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-14T17:55:37.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:37 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:55:37.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:37 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:55:37.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:37 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:55:38.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:38 smithi184 ceph-mon[33536]: pgmap v914: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-14T17:55:38.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:38 smithi178 ceph-mon[39811]: pgmap v914: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-14T17:55:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:38 smithi178 ceph-mon[30321]: pgmap v914: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-14T17:55:41.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:41 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:41.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:41 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:41.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:41 smithi184 ceph-mon[33536]: pgmap v915: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 751 B/s rd, 0 op/s 2024-02-14T17:55:41.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:41 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:41.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:41 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:41.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:41 smithi178 ceph-mon[39811]: pgmap v915: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 751 B/s rd, 0 op/s 2024-02-14T17:55:41.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:41 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:41.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:41 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:41.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:41 smithi178 ceph-mon[30321]: pgmap v915: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 751 B/s rd, 0 op/s 2024-02-14T17:55:41.752 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:55:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:55:42.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:42 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:42.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:42 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:42.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:42 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:55:42.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:42 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:55:42.696 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:42 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:42.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:42 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:42.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:42 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:42 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:55:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:42 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:55:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:42 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:42 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:42 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:42 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:55:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:42 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:55:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:42 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:43 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:43.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:43 smithi184 ceph-mon[33536]: pgmap v916: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:55:43.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:43 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:43.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:43 smithi178 ceph-mon[39811]: pgmap v916: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:55:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:43 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:43 smithi178 ceph-mon[30321]: pgmap v916: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:55:44.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:44 smithi178 ceph-mon[30321]: pgmap v917: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-14T17:55:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:44 smithi178 ceph-mon[39811]: pgmap v917: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-14T17:55:44.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:55:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:44 smithi184 ceph-mon[33536]: pgmap v917: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-14T17:55:46.003 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:55:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:55:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:46.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:46 smithi184 ceph-mon[33536]: pgmap v918: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:55:46.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:46 smithi178 ceph-mon[39811]: pgmap v918: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:55:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:46 smithi178 ceph-mon[30321]: pgmap v918: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:55:48.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:48 smithi184 ceph-mon[33536]: osdmap e596: 8 total, 8 up, 8 in 2024-02-14T17:55:48.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:48.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:48 smithi184 ceph-mon[33536]: pgmap v920: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:55:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:48 smithi178 ceph-mon[30321]: osdmap e596: 8 total, 8 up, 8 in 2024-02-14T17:55:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:48 smithi178 ceph-mon[30321]: pgmap v920: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:55:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:48 smithi178 ceph-mon[39811]: osdmap e596: 8 total, 8 up, 8 in 2024-02-14T17:55:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:48 smithi178 ceph-mon[39811]: pgmap v920: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:55:49.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:49.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:49 smithi184 ceph-mon[33536]: osdmap e597: 8 total, 8 up, 8 in 2024-02-14T17:55:49.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116"}]: dispatch 2024-02-14T17:55:49.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:49 smithi178 ceph-mon[39811]: osdmap e597: 8 total, 8 up, 8 in 2024-02-14T17:55:49.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116"}]: dispatch 2024-02-14T17:55:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:49 smithi178 ceph-mon[30321]: osdmap e597: 8 total, 8 up, 8 in 2024-02-14T17:55:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116"}]: dispatch 2024-02-14T17:55:50.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:50 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:55:50.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116"}]': finished 2024-02-14T17:55:50.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:50 smithi184 ceph-mon[33536]: osdmap e598: 8 total, 8 up, 8 in 2024-02-14T17:55:50.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:50 smithi184 ceph-mon[33536]: pgmap v923: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:55:50.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:50 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:55:50.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116"}]': finished 2024-02-14T17:55:50.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:50 smithi178 ceph-mon[39811]: osdmap e598: 8 total, 8 up, 8 in 2024-02-14T17:55:50.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:50 smithi178 ceph-mon[39811]: pgmap v923: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:55:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:50 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:55:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-116"}]': finished 2024-02-14T17:55:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:50 smithi178 ceph-mon[30321]: osdmap e598: 8 total, 8 up, 8 in 2024-02-14T17:55:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:50 smithi178 ceph-mon[30321]: pgmap v923: 204 pgs: 204 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:55:51.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:51 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:51.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:51 smithi184 ceph-mon[33536]: osdmap e599: 8 total, 8 up, 8 in 2024-02-14T17:55:51.720 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:55:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:55:51.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:51 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:51.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:51 smithi178 ceph-mon[39811]: osdmap e599: 8 total, 8 up, 8 in 2024-02-14T17:55:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:51 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:51 smithi178 ceph-mon[30321]: osdmap e599: 8 total, 8 up, 8 in 2024-02-14T17:55:52.763 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:52 smithi184 ceph-mon[33536]: osdmap e600: 8 total, 8 up, 8 in 2024-02-14T17:55:52.763 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:52.763 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:52 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:52.763 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:52 smithi184 ceph-mon[33536]: pgmap v926: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-14T17:55:52.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:52 smithi178 ceph-mon[39811]: osdmap e600: 8 total, 8 up, 8 in 2024-02-14T17:55:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:52 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:52 smithi178 ceph-mon[39811]: pgmap v926: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-14T17:55:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:52 smithi178 ceph-mon[30321]: osdmap e600: 8 total, 8 up, 8 in 2024-02-14T17:55:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:52 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:52 smithi178 ceph-mon[30321]: pgmap v926: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-14T17:55:53.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:53 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:53.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:53 smithi178 ceph-mon[39811]: osdmap e601: 8 total, 8 up, 8 in 2024-02-14T17:55:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:53 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:53 smithi178 ceph-mon[30321]: osdmap e601: 8 total, 8 up, 8 in 2024-02-14T17:55:53.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:53 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:53 smithi184 ceph-mon[33536]: osdmap e601: 8 total, 8 up, 8 in 2024-02-14T17:55:54.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:54 smithi178 ceph-mon[39811]: osdmap e602: 8 total, 8 up, 8 in 2024-02-14T17:55:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:54 smithi178 ceph-mon[39811]: pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-14T17:55:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:54 smithi178 ceph-mon[30321]: osdmap e602: 8 total, 8 up, 8 in 2024-02-14T17:55:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:54 smithi178 ceph-mon[30321]: pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-14T17:55:54.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:54 smithi184 ceph-mon[33536]: osdmap e602: 8 total, 8 up, 8 in 2024-02-14T17:55:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:55:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:54 smithi184 ceph-mon[33536]: pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-14T17:55:55.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:55:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:55:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:55 smithi178 ceph-mon[39811]: osdmap e603: 8 total, 8 up, 8 in 2024-02-14T17:55:55.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-118"}]: dispatch 2024-02-14T17:55:55.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:55.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:55 smithi178 ceph-mon[30321]: osdmap e603: 8 total, 8 up, 8 in 2024-02-14T17:55:55.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-118"}]: dispatch 2024-02-14T17:55:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:55:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:55 smithi184 ceph-mon[33536]: osdmap e603: 8 total, 8 up, 8 in 2024-02-14T17:55:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-118"}]: dispatch 2024-02-14T17:55:56.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-118"}]': finished 2024-02-14T17:55:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[39811]: osdmap e604: 8 total, 8 up, 8 in 2024-02-14T17:55:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-118", "mode": "writeback"}]: dispatch 2024-02-14T17:55:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:55:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-118", "mode": "writeback"}]': finished 2024-02-14T17:55:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[39811]: osdmap e605: 8 total, 8 up, 8 in 2024-02-14T17:55:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[39811]: pgmap v933: 204 pgs: 204 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 528 B/s wr, 1 op/s 2024-02-14T17:55:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-118"}]': finished 2024-02-14T17:55:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[30321]: osdmap e604: 8 total, 8 up, 8 in 2024-02-14T17:55:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-118", "mode": "writeback"}]: dispatch 2024-02-14T17:55:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:55:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-118", "mode": "writeback"}]': finished 2024-02-14T17:55:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[30321]: osdmap e605: 8 total, 8 up, 8 in 2024-02-14T17:55:56.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:56 smithi178 ceph-mon[30321]: pgmap v933: 204 pgs: 204 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 528 B/s wr, 1 op/s 2024-02-14T17:55:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-118"}]': finished 2024-02-14T17:55:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:56 smithi184 ceph-mon[33536]: osdmap e604: 8 total, 8 up, 8 in 2024-02-14T17:55:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-118", "mode": "writeback"}]: dispatch 2024-02-14T17:55:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:56 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:56 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:55:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-118", "mode": "writeback"}]': finished 2024-02-14T17:55:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:56 smithi184 ceph-mon[33536]: osdmap e605: 8 total, 8 up, 8 in 2024-02-14T17:55:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:56 smithi184 ceph-mon[33536]: pgmap v933: 204 pgs: 204 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 528 B/s wr, 1 op/s 2024-02-14T17:55:58.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:58 smithi178 ceph-mon[39811]: osdmap e606: 8 total, 8 up, 8 in 2024-02-14T17:55:58.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:58 smithi178 ceph-mon[30321]: osdmap e606: 8 total, 8 up, 8 in 2024-02-14T17:55:58.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:58.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:58 smithi184 ceph-mon[33536]: osdmap e606: 8 total, 8 up, 8 in 2024-02-14T17:55:58.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:55:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[39811]: osdmap e607: 8 total, 8 up, 8 in 2024-02-14T17:55:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118"}]: dispatch 2024-02-14T17:55:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[39811]: pgmap v936: 204 pgs: 204 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 529 B/s wr, 1 op/s 2024-02-14T17:55:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:55:59.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118"}]': finished 2024-02-14T17:55:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:59.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[30321]: osdmap e607: 8 total, 8 up, 8 in 2024-02-14T17:55:59.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118"}]: dispatch 2024-02-14T17:55:59.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[30321]: pgmap v936: 204 pgs: 204 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 529 B/s wr, 1 op/s 2024-02-14T17:55:59.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:59.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:55:59.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:55:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118"}]': finished 2024-02-14T17:55:59.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:55:59.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:59 smithi184 ceph-mon[33536]: osdmap e607: 8 total, 8 up, 8 in 2024-02-14T17:55:59.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118"}]: dispatch 2024-02-14T17:55:59.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:59 smithi184 ceph-mon[33536]: pgmap v936: 204 pgs: 204 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 529 B/s wr, 1 op/s 2024-02-14T17:55:59.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:59 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:59.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:59 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:55:59.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:55:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-118"}]': finished 2024-02-14T17:56:00.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:00 smithi178 ceph-mon[39811]: osdmap e608: 8 total, 8 up, 8 in 2024-02-14T17:56:00.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:00 smithi178 ceph-mon[30321]: osdmap e608: 8 total, 8 up, 8 in 2024-02-14T17:56:00.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:00 smithi184 ceph-mon[33536]: osdmap e608: 8 total, 8 up, 8 in 2024-02-14T17:56:01.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:01 smithi178 ceph-mon[39811]: osdmap e609: 8 total, 8 up, 8 in 2024-02-14T17:56:01.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:01 smithi178 ceph-mon[39811]: pgmap v939: 172 pgs: 172 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:01.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:01 smithi178 ceph-mon[30321]: osdmap e609: 8 total, 8 up, 8 in 2024-02-14T17:56:01.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:01 smithi178 ceph-mon[30321]: pgmap v939: 172 pgs: 172 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:01.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:01 smithi184 ceph-mon[33536]: osdmap e609: 8 total, 8 up, 8 in 2024-02-14T17:56:01.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:01 smithi184 ceph-mon[33536]: pgmap v939: 172 pgs: 172 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:01.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:56:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:56:02.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:02 smithi178 ceph-mon[39811]: osdmap e610: 8 total, 8 up, 8 in 2024-02-14T17:56:02.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:02.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:02 smithi178 ceph-mon[30321]: osdmap e610: 8 total, 8 up, 8 in 2024-02-14T17:56:02.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:02.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:02 smithi184 ceph-mon[33536]: osdmap e610: 8 total, 8 up, 8 in 2024-02-14T17:56:02.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:03.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:03 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:03.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:03.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:03 smithi184 ceph-mon[33536]: osdmap e611: 8 total, 8 up, 8 in 2024-02-14T17:56:03.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:03.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:03 smithi184 ceph-mon[33536]: pgmap v942: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:03 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:03 smithi178 ceph-mon[39811]: osdmap e611: 8 total, 8 up, 8 in 2024-02-14T17:56:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:03 smithi178 ceph-mon[39811]: pgmap v942: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:03.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:03 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:03.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:03.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:03 smithi178 ceph-mon[30321]: osdmap e611: 8 total, 8 up, 8 in 2024-02-14T17:56:03.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:03.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:03 smithi178 ceph-mon[30321]: pgmap v942: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:04.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:04.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:04 smithi184 ceph-mon[33536]: osdmap e612: 8 total, 8 up, 8 in 2024-02-14T17:56:04.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-120"}]: dispatch 2024-02-14T17:56:04.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:04.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:04 smithi178 ceph-mon[39811]: osdmap e612: 8 total, 8 up, 8 in 2024-02-14T17:56:04.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-120"}]: dispatch 2024-02-14T17:56:04.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:04.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:04 smithi178 ceph-mon[30321]: osdmap e612: 8 total, 8 up, 8 in 2024-02-14T17:56:04.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-120"}]: dispatch 2024-02-14T17:56:05.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-120"}]': finished 2024-02-14T17:56:05.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:05 smithi184 ceph-mon[33536]: osdmap e613: 8 total, 8 up, 8 in 2024-02-14T17:56:05.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-120", "mode": "writeback"}]: dispatch 2024-02-14T17:56:05.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:05 smithi184 ceph-mon[33536]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:05.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:05 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:05.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-120", "mode": "writeback"}]': finished 2024-02-14T17:56:05.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:05 smithi184 ceph-mon[33536]: osdmap e614: 8 total, 8 up, 8 in 2024-02-14T17:56:05.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-120"}]': finished 2024-02-14T17:56:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[39811]: osdmap e613: 8 total, 8 up, 8 in 2024-02-14T17:56:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-120", "mode": "writeback"}]: dispatch 2024-02-14T17:56:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[39811]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-120", "mode": "writeback"}]': finished 2024-02-14T17:56:05.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[39811]: osdmap e614: 8 total, 8 up, 8 in 2024-02-14T17:56:05.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-120"}]': finished 2024-02-14T17:56:05.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[30321]: osdmap e613: 8 total, 8 up, 8 in 2024-02-14T17:56:05.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-120", "mode": "writeback"}]: dispatch 2024-02-14T17:56:05.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[30321]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-120", "mode": "writeback"}]': finished 2024-02-14T17:56:05.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-mon[30321]: osdmap e614: 8 total, 8 up, 8 in 2024-02-14T17:56:06.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:56:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:56:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:06.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:06.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:06.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:06 smithi184 ceph-mon[33536]: osdmap e615: 8 total, 8 up, 8 in 2024-02-14T17:56:06.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120"}]: dispatch 2024-02-14T17:56:06.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:06.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:06.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:06 smithi178 ceph-mon[30321]: osdmap e615: 8 total, 8 up, 8 in 2024-02-14T17:56:06.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120"}]: dispatch 2024-02-14T17:56:06.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:06.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:06.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:06 smithi178 ceph-mon[39811]: osdmap e615: 8 total, 8 up, 8 in 2024-02-14T17:56:06.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120"}]: dispatch 2024-02-14T17:56:07.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:07 smithi184 ceph-mon[33536]: pgmap v948: 204 pgs: 204 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-14T17:56:07.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:07 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:07.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120"}]': finished 2024-02-14T17:56:07.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:07 smithi184 ceph-mon[33536]: osdmap e616: 8 total, 8 up, 8 in 2024-02-14T17:56:07.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:07 smithi178 ceph-mon[39811]: pgmap v948: 204 pgs: 204 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-14T17:56:07.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:07 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:07.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120"}]': finished 2024-02-14T17:56:07.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:07 smithi178 ceph-mon[39811]: osdmap e616: 8 total, 8 up, 8 in 2024-02-14T17:56:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:07 smithi178 ceph-mon[30321]: pgmap v948: 204 pgs: 204 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-14T17:56:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:07 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-120"}]': finished 2024-02-14T17:56:07.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:07 smithi178 ceph-mon[30321]: osdmap e616: 8 total, 8 up, 8 in 2024-02-14T17:56:09.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:09 smithi178 ceph-mon[30321]: osdmap e617: 8 total, 8 up, 8 in 2024-02-14T17:56:09.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:09 smithi178 ceph-mon[30321]: pgmap v951: 172 pgs: 172 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:56:09.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:09 smithi178 ceph-mon[39811]: osdmap e617: 8 total, 8 up, 8 in 2024-02-14T17:56:09.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:09 smithi178 ceph-mon[39811]: pgmap v951: 172 pgs: 172 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:56:09.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:09 smithi184 ceph-mon[33536]: osdmap e617: 8 total, 8 up, 8 in 2024-02-14T17:56:09.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:09 smithi184 ceph-mon[33536]: pgmap v951: 172 pgs: 172 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:56:10.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:10 smithi178 ceph-mon[30321]: osdmap e618: 8 total, 8 up, 8 in 2024-02-14T17:56:10.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:10 smithi178 ceph-mon[39811]: osdmap e618: 8 total, 8 up, 8 in 2024-02-14T17:56:10.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:10.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:10 smithi184 ceph-mon[33536]: osdmap e618: 8 total, 8 up, 8 in 2024-02-14T17:56:10.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:11.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:11.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:11 smithi178 ceph-mon[30321]: osdmap e619: 8 total, 8 up, 8 in 2024-02-14T17:56:11.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:11.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:11 smithi178 ceph-mon[30321]: pgmap v954: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:11.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:11.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:11 smithi178 ceph-mon[39811]: osdmap e619: 8 total, 8 up, 8 in 2024-02-14T17:56:11.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:11.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:11 smithi178 ceph-mon[39811]: pgmap v954: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:11.369 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:11.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:11 smithi184 ceph-mon[33536]: osdmap e619: 8 total, 8 up, 8 in 2024-02-14T17:56:11.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:11.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:11 smithi184 ceph-mon[33536]: pgmap v954: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:11.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:56:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:56:12.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:12.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:12 smithi178 ceph-mon[30321]: osdmap e620: 8 total, 8 up, 8 in 2024-02-14T17:56:12.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-122"}]: dispatch 2024-02-14T17:56:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:12 smithi178 ceph-mon[39811]: osdmap e620: 8 total, 8 up, 8 in 2024-02-14T17:56:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-122"}]: dispatch 2024-02-14T17:56:12.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:12.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:12 smithi184 ceph-mon[33536]: osdmap e620: 8 total, 8 up, 8 in 2024-02-14T17:56:12.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-122"}]: dispatch 2024-02-14T17:56:13.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:13 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-122"}]': finished 2024-02-14T17:56:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:13 smithi184 ceph-mon[33536]: osdmap e621: 8 total, 8 up, 8 in 2024-02-14T17:56:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-122", "mode": "writeback"}]: dispatch 2024-02-14T17:56:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:13 smithi184 ceph-mon[33536]: pgmap v957: 204 pgs: 204 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:13 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:13.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-122", "mode": "writeback"}]': finished 2024-02-14T17:56:13.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:13.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-122"}]': finished 2024-02-14T17:56:13.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[30321]: osdmap e621: 8 total, 8 up, 8 in 2024-02-14T17:56:13.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-122", "mode": "writeback"}]: dispatch 2024-02-14T17:56:13.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[30321]: pgmap v957: 204 pgs: 204 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:13.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:13.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-122", "mode": "writeback"}]': finished 2024-02-14T17:56:13.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:13.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-122"}]': finished 2024-02-14T17:56:13.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[39811]: osdmap e621: 8 total, 8 up, 8 in 2024-02-14T17:56:13.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-122", "mode": "writeback"}]: dispatch 2024-02-14T17:56:13.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[39811]: pgmap v957: 204 pgs: 204 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:13.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:13.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-122", "mode": "writeback"}]': finished 2024-02-14T17:56:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:14 smithi184 ceph-mon[33536]: osdmap e622: 8 total, 8 up, 8 in 2024-02-14T17:56:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:14 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:14.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:14 smithi178 ceph-mon[30321]: osdmap e622: 8 total, 8 up, 8 in 2024-02-14T17:56:14.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:14.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:14 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:14.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:14 smithi178 ceph-mon[39811]: osdmap e622: 8 total, 8 up, 8 in 2024-02-14T17:56:14.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:14.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:14 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:15 smithi184 ceph-mon[33536]: osdmap e623: 8 total, 8 up, 8 in 2024-02-14T17:56:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122"}]: dispatch 2024-02-14T17:56:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:15 smithi184 ceph-mon[33536]: pgmap v960: 204 pgs: 204 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:15.484 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:15.484 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:15 smithi178 ceph-mon[30321]: osdmap e623: 8 total, 8 up, 8 in 2024-02-14T17:56:15.485 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122"}]: dispatch 2024-02-14T17:56:15.485 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:15 smithi178 ceph-mon[30321]: pgmap v960: 204 pgs: 204 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:15.485 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:15.485 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:15 smithi178 ceph-mon[39811]: osdmap e623: 8 total, 8 up, 8 in 2024-02-14T17:56:15.485 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122"}]: dispatch 2024-02-14T17:56:15.485 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:15 smithi178 ceph-mon[39811]: pgmap v960: 204 pgs: 204 active+clean; 582 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:15.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:56:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:56:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:16.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:16 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:16.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122"}]': finished 2024-02-14T17:56:16.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:16 smithi184 ceph-mon[33536]: osdmap e624: 8 total, 8 up, 8 in 2024-02-14T17:56:16.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:16 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:16.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122"}]': finished 2024-02-14T17:56:16.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:16 smithi178 ceph-mon[39811]: osdmap e624: 8 total, 8 up, 8 in 2024-02-14T17:56:16.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:16 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:16.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-122"}]': finished 2024-02-14T17:56:16.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:16 smithi178 ceph-mon[30321]: osdmap e624: 8 total, 8 up, 8 in 2024-02-14T17:56:17.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:17 smithi184 ceph-mon[33536]: osdmap e625: 8 total, 8 up, 8 in 2024-02-14T17:56:17.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:17 smithi184 ceph-mon[33536]: pgmap v963: 172 pgs: 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:17.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:17 smithi178 ceph-mon[39811]: osdmap e625: 8 total, 8 up, 8 in 2024-02-14T17:56:17.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:17 smithi178 ceph-mon[39811]: pgmap v963: 172 pgs: 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:17.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:17 smithi178 ceph-mon[30321]: osdmap e625: 8 total, 8 up, 8 in 2024-02-14T17:56:17.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:17 smithi178 ceph-mon[30321]: pgmap v963: 172 pgs: 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:18.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:18 smithi184 ceph-mon[33536]: osdmap e626: 8 total, 8 up, 8 in 2024-02-14T17:56:18.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:18.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:18 smithi178 ceph-mon[39811]: osdmap e626: 8 total, 8 up, 8 in 2024-02-14T17:56:18.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:18.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:18 smithi178 ceph-mon[30321]: osdmap e626: 8 total, 8 up, 8 in 2024-02-14T17:56:18.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:19.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:19 smithi184 ceph-mon[33536]: osdmap e627: 8 total, 8 up, 8 in 2024-02-14T17:56:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:19 smithi184 ceph-mon[33536]: pgmap v966: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:19 smithi178 ceph-mon[39811]: osdmap e627: 8 total, 8 up, 8 in 2024-02-14T17:56:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:19 smithi178 ceph-mon[39811]: pgmap v966: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:19 smithi178 ceph-mon[30321]: osdmap e627: 8 total, 8 up, 8 in 2024-02-14T17:56:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:19 smithi178 ceph-mon[30321]: pgmap v966: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:20.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:20 smithi184 ceph-mon[33536]: osdmap e628: 8 total, 8 up, 8 in 2024-02-14T17:56:20.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:20.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:20 smithi178 ceph-mon[39811]: osdmap e628: 8 total, 8 up, 8 in 2024-02-14T17:56:20.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:20.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:20 smithi178 ceph-mon[30321]: osdmap e628: 8 total, 8 up, 8 in 2024-02-14T17:56:20.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:21.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:21.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:21 smithi184 ceph-mon[33536]: osdmap e629: 8 total, 8 up, 8 in 2024-02-14T17:56:21.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-124"}]: dispatch 2024-02-14T17:56:21.472 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:21 smithi184 ceph-mon[33536]: pgmap v969: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:21.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:21.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:21 smithi178 ceph-mon[39811]: osdmap e629: 8 total, 8 up, 8 in 2024-02-14T17:56:21.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-124"}]: dispatch 2024-02-14T17:56:21.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:21 smithi178 ceph-mon[39811]: pgmap v969: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:21 smithi178 ceph-mon[30321]: osdmap e629: 8 total, 8 up, 8 in 2024-02-14T17:56:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-124"}]: dispatch 2024-02-14T17:56:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:21 smithi178 ceph-mon[30321]: pgmap v969: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:21.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:56:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:56:22.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-124"}]': finished 2024-02-14T17:56:22.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:22 smithi184 ceph-mon[33536]: osdmap e630: 8 total, 8 up, 8 in 2024-02-14T17:56:22.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-124", "mode": "writeback"}]: dispatch 2024-02-14T17:56:22.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-124"}]': finished 2024-02-14T17:56:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:22 smithi178 ceph-mon[39811]: osdmap e630: 8 total, 8 up, 8 in 2024-02-14T17:56:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-124", "mode": "writeback"}]: dispatch 2024-02-14T17:56:22.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-124"}]': finished 2024-02-14T17:56:22.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:22 smithi178 ceph-mon[30321]: osdmap e630: 8 total, 8 up, 8 in 2024-02-14T17:56:22.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-124", "mode": "writeback"}]: dispatch 2024-02-14T17:56:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:23 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:23 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-124", "mode": "writeback"}]': finished 2024-02-14T17:56:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:23 smithi184 ceph-mon[33536]: osdmap e631: 8 total, 8 up, 8 in 2024-02-14T17:56:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:23 smithi184 ceph-mon[33536]: pgmap v972: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:56:23.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:23 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:23.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:23 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:23.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-124", "mode": "writeback"}]': finished 2024-02-14T17:56:23.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:23 smithi178 ceph-mon[39811]: osdmap e631: 8 total, 8 up, 8 in 2024-02-14T17:56:23.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:23 smithi178 ceph-mon[39811]: pgmap v972: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:56:23.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:23 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:23.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:23 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:23.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-124", "mode": "writeback"}]': finished 2024-02-14T17:56:23.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:23 smithi178 ceph-mon[30321]: osdmap e631: 8 total, 8 up, 8 in 2024-02-14T17:56:23.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:23 smithi178 ceph-mon[30321]: pgmap v972: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:56:24.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:24 smithi184 ceph-mon[33536]: osdmap e632: 8 total, 8 up, 8 in 2024-02-14T17:56:24.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:24.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:24 smithi178 ceph-mon[39811]: osdmap e632: 8 total, 8 up, 8 in 2024-02-14T17:56:24.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:24.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:24 smithi178 ceph-mon[30321]: osdmap e632: 8 total, 8 up, 8 in 2024-02-14T17:56:24.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:25 smithi184 ceph-mon[33536]: osdmap e633: 8 total, 8 up, 8 in 2024-02-14T17:56:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124"}]: dispatch 2024-02-14T17:56:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:25 smithi184 ceph-mon[33536]: pgmap v975: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:56:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:25 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124"}]': finished 2024-02-14T17:56:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[30321]: osdmap e633: 8 total, 8 up, 8 in 2024-02-14T17:56:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124"}]: dispatch 2024-02-14T17:56:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[30321]: pgmap v975: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:56:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124"}]': finished 2024-02-14T17:56:25.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:25.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[39811]: osdmap e633: 8 total, 8 up, 8 in 2024-02-14T17:56:25.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124"}]: dispatch 2024-02-14T17:56:25.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[39811]: pgmap v975: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:56:25.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:25.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-124"}]': finished 2024-02-14T17:56:26.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:56:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:56:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:26.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:26 smithi184 ceph-mon[33536]: osdmap e634: 8 total, 8 up, 8 in 2024-02-14T17:56:26.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:26 smithi178 ceph-mon[39811]: osdmap e634: 8 total, 8 up, 8 in 2024-02-14T17:56:26.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:26 smithi178 ceph-mon[30321]: osdmap e634: 8 total, 8 up, 8 in 2024-02-14T17:56:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:27 smithi184 ceph-mon[33536]: osdmap e635: 8 total, 8 up, 8 in 2024-02-14T17:56:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:27 smithi184 ceph-mon[33536]: pgmap v978: 172 pgs: 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:27.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:27 smithi178 ceph-mon[30321]: osdmap e635: 8 total, 8 up, 8 in 2024-02-14T17:56:27.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:27 smithi178 ceph-mon[30321]: pgmap v978: 172 pgs: 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:27.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:27 smithi178 ceph-mon[39811]: osdmap e635: 8 total, 8 up, 8 in 2024-02-14T17:56:27.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:27 smithi178 ceph-mon[39811]: pgmap v978: 172 pgs: 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:28.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:28 smithi184 ceph-mon[33536]: osdmap e636: 8 total, 8 up, 8 in 2024-02-14T17:56:28.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:28.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:28 smithi178 ceph-mon[30321]: osdmap e636: 8 total, 8 up, 8 in 2024-02-14T17:56:28.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:28.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:28 smithi178 ceph-mon[39811]: osdmap e636: 8 total, 8 up, 8 in 2024-02-14T17:56:28.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:29 smithi184 ceph-mon[33536]: osdmap e637: 8 total, 8 up, 8 in 2024-02-14T17:56:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:29 smithi184 ceph-mon[33536]: pgmap v981: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:29 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:29 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:29 smithi184 ceph-mon[33536]: osdmap e638: 8 total, 8 up, 8 in 2024-02-14T17:56:29.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-126"}]: dispatch 2024-02-14T17:56:29.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[39811]: osdmap e637: 8 total, 8 up, 8 in 2024-02-14T17:56:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[39811]: pgmap v981: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[39811]: osdmap e638: 8 total, 8 up, 8 in 2024-02-14T17:56:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-126"}]: dispatch 2024-02-14T17:56:29.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:29.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[30321]: osdmap e637: 8 total, 8 up, 8 in 2024-02-14T17:56:29.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:29.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[30321]: pgmap v981: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:29.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:29.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:29.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:29.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[30321]: osdmap e638: 8 total, 8 up, 8 in 2024-02-14T17:56:29.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-126"}]: dispatch 2024-02-14T17:56:31.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-126"}]': finished 2024-02-14T17:56:31.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:31 smithi184 ceph-mon[33536]: osdmap e639: 8 total, 8 up, 8 in 2024-02-14T17:56:31.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-126", "mode": "writeback"}]: dispatch 2024-02-14T17:56:31.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:31 smithi184 ceph-mon[33536]: pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:31.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:31 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:31.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-126", "mode": "writeback"}]': finished 2024-02-14T17:56:31.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:31 smithi184 ceph-mon[33536]: osdmap e640: 8 total, 8 up, 8 in 2024-02-14T17:56:31.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:31.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-126"}]': finished 2024-02-14T17:56:31.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[39811]: osdmap e639: 8 total, 8 up, 8 in 2024-02-14T17:56:31.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-126", "mode": "writeback"}]: dispatch 2024-02-14T17:56:31.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[39811]: pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:31.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:31.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-126", "mode": "writeback"}]': finished 2024-02-14T17:56:31.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[39811]: osdmap e640: 8 total, 8 up, 8 in 2024-02-14T17:56:31.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-126"}]': finished 2024-02-14T17:56:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[30321]: osdmap e639: 8 total, 8 up, 8 in 2024-02-14T17:56:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-126", "mode": "writeback"}]: dispatch 2024-02-14T17:56:31.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[30321]: pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:31.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:31.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-126", "mode": "writeback"}]': finished 2024-02-14T17:56:31.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[30321]: osdmap e640: 8 total, 8 up, 8 in 2024-02-14T17:56:31.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:31.968 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:56:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:56:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:33 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:33 smithi178 ceph-mon[30321]: osdmap e641: 8 total, 8 up, 8 in 2024-02-14T17:56:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126"}]: dispatch 2024-02-14T17:56:33.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:33 smithi178 ceph-mon[30321]: pgmap v987: 204 pgs: 204 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:33.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:33 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:33.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:33.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:33 smithi178 ceph-mon[39811]: osdmap e641: 8 total, 8 up, 8 in 2024-02-14T17:56:33.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126"}]: dispatch 2024-02-14T17:56:33.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:33 smithi178 ceph-mon[39811]: pgmap v987: 204 pgs: 204 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:33.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:33 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:33.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:33.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:33 smithi184 ceph-mon[33536]: osdmap e641: 8 total, 8 up, 8 in 2024-02-14T17:56:33.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126"}]: dispatch 2024-02-14T17:56:33.470 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:33 smithi184 ceph-mon[33536]: pgmap v987: 204 pgs: 204 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:34.064 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:34.064 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9050 ms) 2024-02-14T17:56:34.064 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-02-14T17:56:34.064 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:34.064 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6041 ms) 2024-02-14T17:56:34.064 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5024 ms) 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (24816 ms) 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (13925 ms) 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12062 ms) 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: 00000010 69 73 |is| 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: 00000012 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12044 ms) 2024-02-14T17:56:34.065 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-02-14T17:56:34.066 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:34.066 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6020 ms) 2024-02-14T17:56:34.066 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (558709 ms total) 2024-02-14T17:56:34.066 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: 2024-02-14T17:56:34.066 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-02-14T17:56:34.066 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-02-14T17:56:34.066 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1012 ms) 2024-02-14T17:56:34.066 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-02-14T17:56:34.066 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11061 ms) 2024-02-14T17:56:34.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-02-14T17:56:34.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18109 ms) 2024-02-14T17:56:34.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-02-14T17:56:34.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (1 ms) 2024-02-14T17:56:34.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30183 ms total) 2024-02-14T17:56:34.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: 2024-02-14T17:56:34.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-02-14T17:56:34.067 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7018 ms) 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8033 ms) 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: waiting for scrub... 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: done waiting 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24683 ms) 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9593 ms) 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7985 ms) 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8058 ms) 2024-02-14T17:56:34.068 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-02-14T17:56:34.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10066 ms) 2024-02-14T17:56:34.069 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-02-14T17:56:34.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:34 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:34.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126"}]': finished 2024-02-14T17:56:34.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:34 smithi178 ceph-mon[39811]: osdmap e642: 8 total, 8 up, 8 in 2024-02-14T17:56:34.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:34 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:34.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126"}]': finished 2024-02-14T17:56:34.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:34 smithi178 ceph-mon[30321]: osdmap e642: 8 total, 8 up, 8 in 2024-02-14T17:56:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:34 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-126"}]': finished 2024-02-14T17:56:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:34 smithi184 ceph-mon[33536]: osdmap e642: 8 total, 8 up, 8 in 2024-02-14T17:56:35.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:35 smithi178 ceph-mon[39811]: osdmap e643: 8 total, 8 up, 8 in 2024-02-14T17:56:35.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:35 smithi178 ceph-mon[39811]: pgmap v990: 172 pgs: 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:35.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:35 smithi178 ceph-mon[39811]: osdmap e644: 8 total, 8 up, 8 in 2024-02-14T17:56:35.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:35.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:35 smithi178 ceph-mon[30321]: osdmap e643: 8 total, 8 up, 8 in 2024-02-14T17:56:35.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:35 smithi178 ceph-mon[30321]: pgmap v990: 172 pgs: 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:35.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:35 smithi178 ceph-mon[30321]: osdmap e644: 8 total, 8 up, 8 in 2024-02-14T17:56:35.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:35.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:35 smithi184 ceph-mon[33536]: osdmap e643: 8 total, 8 up, 8 in 2024-02-14T17:56:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:35 smithi184 ceph-mon[33536]: pgmap v990: 172 pgs: 172 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:35 smithi184 ceph-mon[33536]: osdmap e644: 8 total, 8 up, 8 in 2024-02-14T17:56:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:56:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:56:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:37.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:37 smithi178 ceph-mon[39811]: osdmap e645: 8 total, 8 up, 8 in 2024-02-14T17:56:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:37.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:37 smithi178 ceph-mon[39811]: pgmap v993: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:37.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:37.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:37 smithi178 ceph-mon[30321]: osdmap e645: 8 total, 8 up, 8 in 2024-02-14T17:56:37.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:37.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:37 smithi178 ceph-mon[30321]: pgmap v993: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:37.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:37.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:37 smithi184 ceph-mon[33536]: osdmap e645: 8 total, 8 up, 8 in 2024-02-14T17:56:37.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:37.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:37 smithi184 ceph-mon[33536]: pgmap v993: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:38.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:38.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:38 smithi178 ceph-mon[39811]: osdmap e646: 8 total, 8 up, 8 in 2024-02-14T17:56:38.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-128"}]: dispatch 2024-02-14T17:56:38.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:38.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:38 smithi178 ceph-mon[30321]: osdmap e646: 8 total, 8 up, 8 in 2024-02-14T17:56:38.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-128"}]: dispatch 2024-02-14T17:56:38.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:38.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:38 smithi184 ceph-mon[33536]: osdmap e646: 8 total, 8 up, 8 in 2024-02-14T17:56:38.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-128"}]: dispatch 2024-02-14T17:56:39.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-128"}]': finished 2024-02-14T17:56:39.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:39 smithi184 ceph-mon[33536]: osdmap e647: 8 total, 8 up, 8 in 2024-02-14T17:56:39.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-128", "mode": "writeback"}]: dispatch 2024-02-14T17:56:39.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:39 smithi184 ceph-mon[33536]: pgmap v996: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:39.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:39 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:39.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-128", "mode": "writeback"}]': finished 2024-02-14T17:56:39.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-128"}]': finished 2024-02-14T17:56:39.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[39811]: osdmap e647: 8 total, 8 up, 8 in 2024-02-14T17:56:39.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-128", "mode": "writeback"}]: dispatch 2024-02-14T17:56:39.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[39811]: pgmap v996: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:39.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:39.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-128", "mode": "writeback"}]': finished 2024-02-14T17:56:39.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-128"}]': finished 2024-02-14T17:56:39.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[30321]: osdmap e647: 8 total, 8 up, 8 in 2024-02-14T17:56:39.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-128", "mode": "writeback"}]: dispatch 2024-02-14T17:56:39.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[30321]: pgmap v996: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:39.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:39.585 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-128", "mode": "writeback"}]': finished 2024-02-14T17:56:40.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:40 smithi184 ceph-mon[33536]: osdmap e648: 8 total, 8 up, 8 in 2024-02-14T17:56:40.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:40 smithi178 ceph-mon[30321]: osdmap e648: 8 total, 8 up, 8 in 2024-02-14T17:56:40.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:40 smithi178 ceph-mon[39811]: osdmap e648: 8 total, 8 up, 8 in 2024-02-14T17:56:41.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:41 smithi184 ceph-mon[33536]: pgmap v998: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:41.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:41 smithi178 ceph-mon[30321]: pgmap v998: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:41.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:41 smithi178 ceph-mon[39811]: pgmap v998: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:41.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:56:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:56:42.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:42 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:56:42.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:42 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:56:42.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:42 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:56:43.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:43 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:43.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:43 smithi184 ceph-mon[33536]: pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-14T17:56:43.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:56:43.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:56:43.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:43 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:43.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:43 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:43.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:43 smithi178 ceph-mon[39811]: pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-14T17:56:43.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:56:43.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:56:43.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:43 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:43 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:43 smithi178 ceph-mon[30321]: pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-14T17:56:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:56:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:56:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:43 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:44 smithi184 ceph-mon[33536]: pgmap v1000: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 841 B/s rd, 280 B/s wr, 1 op/s 2024-02-14T17:56:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:45.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:44 smithi178 ceph-mon[39811]: pgmap v1000: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 841 B/s rd, 280 B/s wr, 1 op/s 2024-02-14T17:56:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:44 smithi178 ceph-mon[30321]: pgmap v1000: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 841 B/s rd, 280 B/s wr, 1 op/s 2024-02-14T17:56:45.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:45.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:45 smithi184 ceph-mon[33536]: osdmap e649: 8 total, 8 up, 8 in 2024-02-14T17:56:45.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128"}]: dispatch 2024-02-14T17:56:46.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:56:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:56:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:45 smithi178 ceph-mon[30321]: osdmap e649: 8 total, 8 up, 8 in 2024-02-14T17:56:46.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128"}]: dispatch 2024-02-14T17:56:46.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:46.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:45 smithi178 ceph-mon[39811]: osdmap e649: 8 total, 8 up, 8 in 2024-02-14T17:56:46.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128"}]: dispatch 2024-02-14T17:56:46.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:46 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:46.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128"}]': finished 2024-02-14T17:56:46.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:46 smithi184 ceph-mon[33536]: osdmap e650: 8 total, 8 up, 8 in 2024-02-14T17:56:46.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:46 smithi184 ceph-mon[33536]: pgmap v1003: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 280 B/s wr, 2 op/s 2024-02-14T17:56:47.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:46 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128"}]': finished 2024-02-14T17:56:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:46 smithi178 ceph-mon[39811]: osdmap e650: 8 total, 8 up, 8 in 2024-02-14T17:56:47.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:46 smithi178 ceph-mon[39811]: pgmap v1003: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 280 B/s wr, 2 op/s 2024-02-14T17:56:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:46 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-128"}]': finished 2024-02-14T17:56:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:46 smithi178 ceph-mon[30321]: osdmap e650: 8 total, 8 up, 8 in 2024-02-14T17:56:47.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:46 smithi178 ceph-mon[30321]: pgmap v1003: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 280 B/s wr, 2 op/s 2024-02-14T17:56:47.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:47 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:47 smithi184 ceph-mon[33536]: osdmap e651: 8 total, 8 up, 8 in 2024-02-14T17:56:48.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:47 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:48.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:47 smithi178 ceph-mon[39811]: osdmap e651: 8 total, 8 up, 8 in 2024-02-14T17:56:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:47 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:48.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:47 smithi178 ceph-mon[30321]: osdmap e651: 8 total, 8 up, 8 in 2024-02-14T17:56:48.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:48 smithi184 ceph-mon[33536]: osdmap e652: 8 total, 8 up, 8 in 2024-02-14T17:56:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:48.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:48 smithi184 ceph-mon[33536]: pgmap v1006: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:56:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:48 smithi178 ceph-mon[39811]: osdmap e652: 8 total, 8 up, 8 in 2024-02-14T17:56:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:49.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:48 smithi178 ceph-mon[39811]: pgmap v1006: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:56:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:48 smithi178 ceph-mon[30321]: osdmap e652: 8 total, 8 up, 8 in 2024-02-14T17:56:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:49.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:48 smithi178 ceph-mon[30321]: pgmap v1006: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:56:49.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:49.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:49 smithi184 ceph-mon[33536]: osdmap e653: 8 total, 8 up, 8 in 2024-02-14T17:56:49.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:49 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:50.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:50.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:49 smithi178 ceph-mon[39811]: osdmap e653: 8 total, 8 up, 8 in 2024-02-14T17:56:50.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:49 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:49 smithi178 ceph-mon[30321]: osdmap e653: 8 total, 8 up, 8 in 2024-02-14T17:56:50.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:49 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:50 smithi184 ceph-mon[33536]: osdmap e654: 8 total, 8 up, 8 in 2024-02-14T17:56:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-130"}]: dispatch 2024-02-14T17:56:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:50 smithi184 ceph-mon[33536]: pgmap v1009: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:51.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:50 smithi178 ceph-mon[39811]: osdmap e654: 8 total, 8 up, 8 in 2024-02-14T17:56:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-130"}]: dispatch 2024-02-14T17:56:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:50 smithi178 ceph-mon[39811]: pgmap v1009: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:50 smithi178 ceph-mon[30321]: osdmap e654: 8 total, 8 up, 8 in 2024-02-14T17:56:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-130"}]: dispatch 2024-02-14T17:56:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:50 smithi178 ceph-mon[30321]: pgmap v1009: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:51.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:56:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:56:51.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-130"}]': finished 2024-02-14T17:56:51.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:51 smithi184 ceph-mon[33536]: osdmap e655: 8 total, 8 up, 8 in 2024-02-14T17:56:51.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-130", "mode": "writeback"}]: dispatch 2024-02-14T17:56:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-130"}]': finished 2024-02-14T17:56:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:51 smithi178 ceph-mon[30321]: osdmap e655: 8 total, 8 up, 8 in 2024-02-14T17:56:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-130", "mode": "writeback"}]: dispatch 2024-02-14T17:56:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-130"}]': finished 2024-02-14T17:56:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:51 smithi178 ceph-mon[39811]: osdmap e655: 8 total, 8 up, 8 in 2024-02-14T17:56:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-130", "mode": "writeback"}]: dispatch 2024-02-14T17:56:52.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:52 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:52 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-130", "mode": "writeback"}]': finished 2024-02-14T17:56:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:52 smithi184 ceph-mon[33536]: osdmap e656: 8 total, 8 up, 8 in 2024-02-14T17:56:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:52 smithi184 ceph-mon[33536]: pgmap v1012: 204 pgs: 204 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-130", "mode": "writeback"}]': finished 2024-02-14T17:56:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[39811]: osdmap e656: 8 total, 8 up, 8 in 2024-02-14T17:56:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[39811]: pgmap v1012: 204 pgs: 204 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-130", "mode": "writeback"}]': finished 2024-02-14T17:56:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[30321]: osdmap e656: 8 total, 8 up, 8 in 2024-02-14T17:56:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:56:53.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:52 smithi178 ceph-mon[30321]: pgmap v1012: 204 pgs: 204 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:53 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:53 smithi184 ceph-mon[33536]: osdmap e657: 8 total, 8 up, 8 in 2024-02-14T17:56:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:53 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130"}]: dispatch 2024-02-14T17:56:54.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:53 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:54.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:53 smithi178 ceph-mon[39811]: osdmap e657: 8 total, 8 up, 8 in 2024-02-14T17:56:54.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:53 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130"}]: dispatch 2024-02-14T17:56:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:53 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:56:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:53 smithi178 ceph-mon[30321]: osdmap e657: 8 total, 8 up, 8 in 2024-02-14T17:56:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:53 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130"}]: dispatch 2024-02-14T17:56:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:54 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130"}]': finished 2024-02-14T17:56:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:54 smithi184 ceph-mon[33536]: osdmap e658: 8 total, 8 up, 8 in 2024-02-14T17:56:54.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:54 smithi184 ceph-mon[33536]: pgmap v1015: 204 pgs: 204 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:55.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:54 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:55.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130"}]': finished 2024-02-14T17:56:55.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:54 smithi178 ceph-mon[39811]: osdmap e658: 8 total, 8 up, 8 in 2024-02-14T17:56:55.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:54 smithi178 ceph-mon[39811]: pgmap v1015: 204 pgs: 204 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:55.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:54 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:56:55.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-130"}]': finished 2024-02-14T17:56:55.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:54 smithi178 ceph-mon[30321]: osdmap e658: 8 total, 8 up, 8 in 2024-02-14T17:56:55.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:54 smithi178 ceph-mon[30321]: pgmap v1015: 204 pgs: 204 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:55 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:55 smithi184 ceph-mon[33536]: osdmap e659: 8 total, 8 up, 8 in 2024-02-14T17:56:56.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:55 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:56.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:55 smithi178 ceph-mon[39811]: osdmap e659: 8 total, 8 up, 8 in 2024-02-14T17:56:56.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:55 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:56.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:55 smithi178 ceph-mon[30321]: osdmap e659: 8 total, 8 up, 8 in 2024-02-14T17:56:56.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:56:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:56:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:56 smithi184 ceph-mon[33536]: osdmap e660: 8 total, 8 up, 8 in 2024-02-14T17:56:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:56 smithi184 ceph-mon[33536]: pgmap v1018: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:57.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:56 smithi178 ceph-mon[39811]: osdmap e660: 8 total, 8 up, 8 in 2024-02-14T17:56:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:56 smithi178 ceph-mon[39811]: pgmap v1018: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:56 smithi178 ceph-mon[30321]: osdmap e660: 8 total, 8 up, 8 in 2024-02-14T17:56:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:56 smithi178 ceph-mon[30321]: pgmap v1018: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:57.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:57.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:57 smithi184 ceph-mon[33536]: osdmap e661: 8 total, 8 up, 8 in 2024-02-14T17:56:57.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:58.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:57 smithi178 ceph-mon[39811]: osdmap e661: 8 total, 8 up, 8 in 2024-02-14T17:56:58.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:57 smithi178 ceph-mon[30321]: osdmap e661: 8 total, 8 up, 8 in 2024-02-14T17:56:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:56:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:58 smithi184 ceph-mon[33536]: osdmap e662: 8 total, 8 up, 8 in 2024-02-14T17:56:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]: dispatch 2024-02-14T17:56:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:58 smithi184 ceph-mon[33536]: pgmap v1021: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:58 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:58 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]': finished 2024-02-14T17:56:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:58 smithi184 ceph-mon[33536]: osdmap e663: 8 total, 8 up, 8 in 2024-02-14T17:56:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[39811]: osdmap e662: 8 total, 8 up, 8 in 2024-02-14T17:56:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]: dispatch 2024-02-14T17:56:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[39811]: pgmap v1021: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]': finished 2024-02-14T17:56:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[39811]: osdmap e663: 8 total, 8 up, 8 in 2024-02-14T17:56:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:56:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[30321]: osdmap e662: 8 total, 8 up, 8 in 2024-02-14T17:56:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]: dispatch 2024-02-14T17:56:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[30321]: pgmap v1021: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 703 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:56:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:56:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]': finished 2024-02-14T17:56:59.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:58 smithi178 ceph-mon[30321]: osdmap e663: 8 total, 8 up, 8 in 2024-02-14T17:56:59.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-132", "mode": "writeback"}]: dispatch 2024-02-14T17:56:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:59 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:56:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-132", "mode": "writeback"}]': finished 2024-02-14T17:56:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:56:59 smithi184 ceph-mon[33536]: osdmap e664: 8 total, 8 up, 8 in 2024-02-14T17:57:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-132", "mode": "writeback"}]: dispatch 2024-02-14T17:57:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:59 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-132", "mode": "writeback"}]': finished 2024-02-14T17:57:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:56:59 smithi178 ceph-mon[39811]: osdmap e664: 8 total, 8 up, 8 in 2024-02-14T17:57:00.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-132", "mode": "writeback"}]: dispatch 2024-02-14T17:57:00.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:59 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:00.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-132", "mode": "writeback"}]': finished 2024-02-14T17:57:00.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:56:59 smithi178 ceph-mon[30321]: osdmap e664: 8 total, 8 up, 8 in 2024-02-14T17:57:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:00 smithi178 ceph-mon[39811]: pgmap v1024: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 703 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:00 smithi178 ceph-mon[30321]: pgmap v1024: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 703 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:01.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:00 smithi184 ceph-mon[33536]: pgmap v1024: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 703 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:01.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:57:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:57:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:01 smithi184 ceph-mon[33536]: osdmap e665: 8 total, 8 up, 8 in 2024-02-14T17:57:01.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:01 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:02.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:01 smithi178 ceph-mon[39811]: osdmap e665: 8 total, 8 up, 8 in 2024-02-14T17:57:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:01 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:01 smithi178 ceph-mon[30321]: osdmap e665: 8 total, 8 up, 8 in 2024-02-14T17:57:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:01 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:03.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:02 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:02 smithi178 ceph-mon[30321]: osdmap e666: 8 total, 8 up, 8 in 2024-02-14T17:57:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:02 smithi178 ceph-mon[30321]: pgmap v1027: 204 pgs: 204 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:02 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:02 smithi178 ceph-mon[39811]: osdmap e666: 8 total, 8 up, 8 in 2024-02-14T17:57:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:02 smithi178 ceph-mon[39811]: pgmap v1027: 204 pgs: 204 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:03.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:02 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:03.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:02 smithi184 ceph-mon[33536]: osdmap e666: 8 total, 8 up, 8 in 2024-02-14T17:57:03.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:03.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:02 smithi184 ceph-mon[33536]: pgmap v1027: 204 pgs: 204 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:04.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:03 smithi178 ceph-mon[39811]: osdmap e667: 8 total, 8 up, 8 in 2024-02-14T17:57:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]: dispatch 2024-02-14T17:57:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:03 smithi178 ceph-mon[30321]: osdmap e667: 8 total, 8 up, 8 in 2024-02-14T17:57:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]: dispatch 2024-02-14T17:57:04.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:04.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:03 smithi184 ceph-mon[33536]: osdmap e667: 8 total, 8 up, 8 in 2024-02-14T17:57:04.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]: dispatch 2024-02-14T17:57:05.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]': finished 2024-02-14T17:57:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:04 smithi178 ceph-mon[39811]: osdmap e668: 8 total, 8 up, 8 in 2024-02-14T17:57:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:04 smithi178 ceph-mon[39811]: pgmap v1030: 204 pgs: 204 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:04 smithi178 ceph-mon[39811]: osdmap e669: 8 total, 8 up, 8 in 2024-02-14T17:57:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]': finished 2024-02-14T17:57:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:04 smithi178 ceph-mon[30321]: osdmap e668: 8 total, 8 up, 8 in 2024-02-14T17:57:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:04 smithi178 ceph-mon[30321]: pgmap v1030: 204 pgs: 204 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:04 smithi178 ceph-mon[30321]: osdmap e669: 8 total, 8 up, 8 in 2024-02-14T17:57:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-132"}]': finished 2024-02-14T17:57:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:04 smithi184 ceph-mon[33536]: osdmap e668: 8 total, 8 up, 8 in 2024-02-14T17:57:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:04 smithi184 ceph-mon[33536]: pgmap v1030: 204 pgs: 204 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:04 smithi184 ceph-mon[33536]: osdmap e669: 8 total, 8 up, 8 in 2024-02-14T17:57:06.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:06.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:06.085 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:57:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:57:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:06.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:07.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:07.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:06 smithi178 ceph-mon[39811]: osdmap e670: 8 total, 8 up, 8 in 2024-02-14T17:57:07.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132"}]: dispatch 2024-02-14T17:57:07.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:06 smithi178 ceph-mon[39811]: pgmap v1033: 204 pgs: 204 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:57:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:06 smithi178 ceph-mon[30321]: osdmap e670: 8 total, 8 up, 8 in 2024-02-14T17:57:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132"}]: dispatch 2024-02-14T17:57:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:06 smithi178 ceph-mon[30321]: pgmap v1033: 204 pgs: 204 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:57:07.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:07.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:06 smithi184 ceph-mon[33536]: osdmap e670: 8 total, 8 up, 8 in 2024-02-14T17:57:07.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132"}]: dispatch 2024-02-14T17:57:07.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:06 smithi184 ceph-mon[33536]: pgmap v1033: 204 pgs: 204 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:57:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:07 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132"}]': finished 2024-02-14T17:57:08.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:07 smithi178 ceph-mon[39811]: osdmap e671: 8 total, 8 up, 8 in 2024-02-14T17:57:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:07 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132"}]': finished 2024-02-14T17:57:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:07 smithi178 ceph-mon[30321]: osdmap e671: 8 total, 8 up, 8 in 2024-02-14T17:57:08.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:07 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:08.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-132"}]': finished 2024-02-14T17:57:08.220 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:07 smithi184 ceph-mon[33536]: osdmap e671: 8 total, 8 up, 8 in 2024-02-14T17:57:09.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:08 smithi178 ceph-mon[39811]: osdmap e672: 8 total, 8 up, 8 in 2024-02-14T17:57:09.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:08 smithi178 ceph-mon[39811]: pgmap v1036: 172 pgs: 172 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:57:09.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:08 smithi178 ceph-mon[30321]: osdmap e672: 8 total, 8 up, 8 in 2024-02-14T17:57:09.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:08 smithi178 ceph-mon[30321]: pgmap v1036: 172 pgs: 172 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:57:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:08 smithi184 ceph-mon[33536]: osdmap e672: 8 total, 8 up, 8 in 2024-02-14T17:57:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:08 smithi184 ceph-mon[33536]: pgmap v1036: 172 pgs: 172 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:57:10.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:09 smithi178 ceph-mon[39811]: osdmap e673: 8 total, 8 up, 8 in 2024-02-14T17:57:10.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:10.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:09 smithi178 ceph-mon[30321]: osdmap e673: 8 total, 8 up, 8 in 2024-02-14T17:57:10.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:10.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:09 smithi184 ceph-mon[33536]: osdmap e673: 8 total, 8 up, 8 in 2024-02-14T17:57:10.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:10 smithi178 ceph-mon[39811]: osdmap e674: 8 total, 8 up, 8 in 2024-02-14T17:57:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:10 smithi178 ceph-mon[39811]: pgmap v1039: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:10 smithi178 ceph-mon[30321]: osdmap e674: 8 total, 8 up, 8 in 2024-02-14T17:57:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:11.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:10 smithi178 ceph-mon[30321]: pgmap v1039: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:11.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:11.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:10 smithi184 ceph-mon[33536]: osdmap e674: 8 total, 8 up, 8 in 2024-02-14T17:57:11.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:11.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:10 smithi184 ceph-mon[33536]: pgmap v1039: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:11.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:57:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:57:12.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:12.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[39811]: osdmap e675: 8 total, 8 up, 8 in 2024-02-14T17:57:12.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-134"}]: dispatch 2024-02-14T17:57:12.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-134"}]': finished 2024-02-14T17:57:12.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[39811]: osdmap e676: 8 total, 8 up, 8 in 2024-02-14T17:57:12.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-134", "mode": "writeback"}]: dispatch 2024-02-14T17:57:12.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:12.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[30321]: osdmap e675: 8 total, 8 up, 8 in 2024-02-14T17:57:12.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-134"}]: dispatch 2024-02-14T17:57:12.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-134"}]': finished 2024-02-14T17:57:12.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[30321]: osdmap e676: 8 total, 8 up, 8 in 2024-02-14T17:57:12.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:11 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-134", "mode": "writeback"}]: dispatch 2024-02-14T17:57:12.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:12.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:11 smithi184 ceph-mon[33536]: osdmap e675: 8 total, 8 up, 8 in 2024-02-14T17:57:12.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-134"}]: dispatch 2024-02-14T17:57:12.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-134"}]': finished 2024-02-14T17:57:12.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:11 smithi184 ceph-mon[33536]: osdmap e676: 8 total, 8 up, 8 in 2024-02-14T17:57:12.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:11 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-134", "mode": "writeback"}]: dispatch 2024-02-14T17:57:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-134", "mode": "writeback"}]': finished 2024-02-14T17:57:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[39811]: osdmap e677: 8 total, 8 up, 8 in 2024-02-14T17:57:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[39811]: pgmap v1043: 204 pgs: 204 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:57:13.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:13.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:13.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-134", "mode": "writeback"}]': finished 2024-02-14T17:57:13.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[30321]: osdmap e677: 8 total, 8 up, 8 in 2024-02-14T17:57:13.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:13.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:12 smithi178 ceph-mon[30321]: pgmap v1043: 204 pgs: 204 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:57:13.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:12 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:13.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:12 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:13.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-134", "mode": "writeback"}]': finished 2024-02-14T17:57:13.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:12 smithi184 ceph-mon[33536]: osdmap e677: 8 total, 8 up, 8 in 2024-02-14T17:57:13.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:12 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:13.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:12 smithi184 ceph-mon[33536]: pgmap v1043: 204 pgs: 204 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T17:57:14.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:14.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:14 smithi178 ceph-mon[39811]: osdmap e678: 8 total, 8 up, 8 in 2024-02-14T17:57:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134"}]: dispatch 2024-02-14T17:57:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:14 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:14.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:14 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:14 smithi178 ceph-mon[30321]: osdmap e678: 8 total, 8 up, 8 in 2024-02-14T17:57:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134"}]: dispatch 2024-02-14T17:57:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:14 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:14.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:14 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:14 smithi184 ceph-mon[33536]: osdmap e678: 8 total, 8 up, 8 in 2024-02-14T17:57:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134"}]: dispatch 2024-02-14T17:57:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:14 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:14.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:14 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:15 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134"}]': finished 2024-02-14T17:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:15 smithi178 ceph-mon[39811]: osdmap e679: 8 total, 8 up, 8 in 2024-02-14T17:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:15 smithi178 ceph-mon[39811]: pgmap v1046: 204 pgs: 204 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:15 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134"}]': finished 2024-02-14T17:57:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:15 smithi178 ceph-mon[30321]: osdmap e679: 8 total, 8 up, 8 in 2024-02-14T17:57:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:15 smithi178 ceph-mon[30321]: pgmap v1046: 204 pgs: 204 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:15.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:15 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-134"}]': finished 2024-02-14T17:57:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:15 smithi184 ceph-mon[33536]: osdmap e679: 8 total, 8 up, 8 in 2024-02-14T17:57:15.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:15 smithi184 ceph-mon[33536]: pgmap v1046: 204 pgs: 204 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:16.077 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:57:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:57:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:16.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:16 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:16.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:16 smithi178 ceph-mon[39811]: osdmap e680: 8 total, 8 up, 8 in 2024-02-14T17:57:16.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:16 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:16.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:16 smithi178 ceph-mon[30321]: osdmap e680: 8 total, 8 up, 8 in 2024-02-14T17:57:16.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:16 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:16.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:16 smithi184 ceph-mon[33536]: osdmap e680: 8 total, 8 up, 8 in 2024-02-14T17:57:17.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:17 smithi184 ceph-mon[33536]: osdmap e681: 8 total, 8 up, 8 in 2024-02-14T17:57:17.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:17.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:17 smithi184 ceph-mon[33536]: pgmap v1049: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:57:17.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:17 smithi178 ceph-mon[39811]: osdmap e681: 8 total, 8 up, 8 in 2024-02-14T17:57:17.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:17.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:17 smithi178 ceph-mon[39811]: pgmap v1049: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:57:17.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:17 smithi178 ceph-mon[30321]: osdmap e681: 8 total, 8 up, 8 in 2024-02-14T17:57:17.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:17.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:17 smithi178 ceph-mon[30321]: pgmap v1049: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:57:18.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:18.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:18 smithi184 ceph-mon[33536]: osdmap e682: 8 total, 8 up, 8 in 2024-02-14T17:57:18.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:18 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:18.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:18.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:18 smithi178 ceph-mon[39811]: osdmap e682: 8 total, 8 up, 8 in 2024-02-14T17:57:18.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:18 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:18.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:18.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:18 smithi178 ceph-mon[30321]: osdmap e682: 8 total, 8 up, 8 in 2024-02-14T17:57:18.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:18 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:19 smithi184 ceph-mon[33536]: osdmap e683: 8 total, 8 up, 8 in 2024-02-14T17:57:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-136"}]: dispatch 2024-02-14T17:57:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:19 smithi184 ceph-mon[33536]: pgmap v1052: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:57:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-136"}]': finished 2024-02-14T17:57:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:19 smithi184 ceph-mon[33536]: osdmap e684: 8 total, 8 up, 8 in 2024-02-14T17:57:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-136", "mode": "writeback"}]: dispatch 2024-02-14T17:57:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[39811]: osdmap e683: 8 total, 8 up, 8 in 2024-02-14T17:57:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-136"}]: dispatch 2024-02-14T17:57:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[39811]: pgmap v1052: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:57:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-136"}]': finished 2024-02-14T17:57:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[39811]: osdmap e684: 8 total, 8 up, 8 in 2024-02-14T17:57:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-136", "mode": "writeback"}]: dispatch 2024-02-14T17:57:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[30321]: osdmap e683: 8 total, 8 up, 8 in 2024-02-14T17:57:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-136"}]: dispatch 2024-02-14T17:57:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[30321]: pgmap v1052: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:57:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-136"}]': finished 2024-02-14T17:57:19.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[30321]: osdmap e684: 8 total, 8 up, 8 in 2024-02-14T17:57:19.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-136", "mode": "writeback"}]: dispatch 2024-02-14T17:57:20.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:20 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:20.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-136", "mode": "writeback"}]': finished 2024-02-14T17:57:20.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:20 smithi184 ceph-mon[33536]: osdmap e685: 8 total, 8 up, 8 in 2024-02-14T17:57:20.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:20 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:20.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-136", "mode": "writeback"}]': finished 2024-02-14T17:57:20.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:20 smithi178 ceph-mon[39811]: osdmap e685: 8 total, 8 up, 8 in 2024-02-14T17:57:20.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:20 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:20.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-136", "mode": "writeback"}]': finished 2024-02-14T17:57:20.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:20 smithi178 ceph-mon[30321]: osdmap e685: 8 total, 8 up, 8 in 2024-02-14T17:57:21.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:21.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:21 smithi184 ceph-mon[33536]: pgmap v1055: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:21.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:21 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:21.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:21.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:21 smithi178 ceph-mon[39811]: pgmap v1055: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:21.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:21 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:21 smithi178 ceph-mon[30321]: pgmap v1055: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:21 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:21.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:57:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:57:22.399 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:22.400 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:22 smithi184 ceph-mon[33536]: osdmap e686: 8 total, 8 up, 8 in 2024-02-14T17:57:22.400 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136"}]: dispatch 2024-02-14T17:57:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:22 smithi178 ceph-mon[39811]: osdmap e686: 8 total, 8 up, 8 in 2024-02-14T17:57:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136"}]: dispatch 2024-02-14T17:57:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:57:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:22 smithi178 ceph-mon[30321]: osdmap e686: 8 total, 8 up, 8 in 2024-02-14T17:57:22.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136"}]: dispatch 2024-02-14T17:57:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:23 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:23 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136"}]': finished 2024-02-14T17:57:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:23 smithi184 ceph-mon[33536]: osdmap e687: 8 total, 8 up, 8 in 2024-02-14T17:57:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:23 smithi184 ceph-mon[33536]: pgmap v1058: 204 pgs: 204 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-14T17:57:23.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:23 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:23.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:23 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:23.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136"}]': finished 2024-02-14T17:57:23.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:23 smithi178 ceph-mon[39811]: osdmap e687: 8 total, 8 up, 8 in 2024-02-14T17:57:23.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:23 smithi178 ceph-mon[39811]: pgmap v1058: 204 pgs: 204 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-14T17:57:23.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:23 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:23.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:23 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:23.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-136"}]': finished 2024-02-14T17:57:23.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:23 smithi178 ceph-mon[30321]: osdmap e687: 8 total, 8 up, 8 in 2024-02-14T17:57:23.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:23 smithi178 ceph-mon[30321]: pgmap v1058: 204 pgs: 204 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-14T17:57:24.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:24 smithi184 ceph-mon[33536]: osdmap e688: 8 total, 8 up, 8 in 2024-02-14T17:57:24.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:24 smithi178 ceph-mon[39811]: osdmap e688: 8 total, 8 up, 8 in 2024-02-14T17:57:24.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:24 smithi178 ceph-mon[30321]: osdmap e688: 8 total, 8 up, 8 in 2024-02-14T17:57:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:25 smithi184 ceph-mon[33536]: osdmap e689: 8 total, 8 up, 8 in 2024-02-14T17:57:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:25.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:25 smithi184 ceph-mon[33536]: pgmap v1061: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-14T17:57:25.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:25 smithi178 ceph-mon[39811]: osdmap e689: 8 total, 8 up, 8 in 2024-02-14T17:57:25.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:25.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:25 smithi178 ceph-mon[39811]: pgmap v1061: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-14T17:57:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:25 smithi178 ceph-mon[30321]: osdmap e689: 8 total, 8 up, 8 in 2024-02-14T17:57:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:25.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:25 smithi178 ceph-mon[30321]: pgmap v1061: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-14T17:57:26.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:57:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:57:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:26.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:26.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:26 smithi184 ceph-mon[33536]: osdmap e690: 8 total, 8 up, 8 in 2024-02-14T17:57:26.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:26.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:26 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:26.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:26.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:26 smithi178 ceph-mon[30321]: osdmap e690: 8 total, 8 up, 8 in 2024-02-14T17:57:26.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:26.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:26 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:26.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:26.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:26 smithi178 ceph-mon[39811]: osdmap e690: 8 total, 8 up, 8 in 2024-02-14T17:57:26.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:26.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:26 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:27.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:27 smithi184 ceph-mon[33536]: osdmap e691: 8 total, 8 up, 8 in 2024-02-14T17:57:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:27 smithi184 ceph-mon[33536]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:27.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:27.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:27.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:27 smithi178 ceph-mon[30321]: osdmap e691: 8 total, 8 up, 8 in 2024-02-14T17:57:27.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:27.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:27 smithi178 ceph-mon[30321]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:27.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:27.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:27.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:27 smithi178 ceph-mon[39811]: osdmap e691: 8 total, 8 up, 8 in 2024-02-14T17:57:27.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:27.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:27 smithi178 ceph-mon[39811]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:27.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:28.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:28 smithi184 ceph-mon[33536]: osdmap e692: 8 total, 8 up, 8 in 2024-02-14T17:57:28.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:28.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:28.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:28 smithi184 ceph-mon[33536]: osdmap e693: 8 total, 8 up, 8 in 2024-02-14T17:57:28.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:28.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:28 smithi178 ceph-mon[30321]: osdmap e692: 8 total, 8 up, 8 in 2024-02-14T17:57:28.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:28.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:28.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:28 smithi178 ceph-mon[30321]: osdmap e693: 8 total, 8 up, 8 in 2024-02-14T17:57:28.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:28.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:28 smithi178 ceph-mon[39811]: osdmap e692: 8 total, 8 up, 8 in 2024-02-14T17:57:28.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:28.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:28.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:28 smithi178 ceph-mon[39811]: osdmap e693: 8 total, 8 up, 8 in 2024-02-14T17:57:28.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:29.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:29 smithi184 ceph-mon[33536]: pgmap v1067: 204 pgs: 204 active+clean; 4.4 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:29 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:29 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:57:29.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:29 smithi184 ceph-mon[33536]: osdmap e694: 8 total, 8 up, 8 in 2024-02-14T17:57:29.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:29 smithi178 ceph-mon[39811]: pgmap v1067: 204 pgs: 204 active+clean; 4.4 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:29 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:29 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:57:29.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:29 smithi178 ceph-mon[39811]: osdmap e694: 8 total, 8 up, 8 in 2024-02-14T17:57:29.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:29 smithi178 ceph-mon[30321]: pgmap v1067: 204 pgs: 204 active+clean; 4.4 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:29.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:29 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:29.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:29 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:29.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:57:29.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:29 smithi178 ceph-mon[30321]: osdmap e694: 8 total, 8 up, 8 in 2024-02-14T17:57:30.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:30.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138"}]: dispatch 2024-02-14T17:57:30.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:30.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138"}]: dispatch 2024-02-14T17:57:30.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:30.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138"}]: dispatch 2024-02-14T17:57:31.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138"}]': finished 2024-02-14T17:57:31.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:31 smithi178 ceph-mon[39811]: osdmap e695: 8 total, 8 up, 8 in 2024-02-14T17:57:31.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:31 smithi178 ceph-mon[39811]: pgmap v1070: 204 pgs: 204 active+clean; 4.4 MiB data, 760 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:57:31.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:31 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138"}]': finished 2024-02-14T17:57:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:31 smithi178 ceph-mon[30321]: osdmap e695: 8 total, 8 up, 8 in 2024-02-14T17:57:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:31 smithi178 ceph-mon[30321]: pgmap v1070: 204 pgs: 204 active+clean; 4.4 MiB data, 760 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:57:31.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:31 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:31.598 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-138"}]': finished 2024-02-14T17:57:31.599 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:31 smithi184 ceph-mon[33536]: osdmap e695: 8 total, 8 up, 8 in 2024-02-14T17:57:31.599 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:31 smithi184 ceph-mon[33536]: pgmap v1070: 204 pgs: 204 active+clean; 4.4 MiB data, 760 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:57:31.599 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:31 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:31.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:57:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:57:32.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:32 smithi178 ceph-mon[39811]: osdmap e696: 8 total, 8 up, 8 in 2024-02-14T17:57:32.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:32 smithi178 ceph-mon[30321]: osdmap e696: 8 total, 8 up, 8 in 2024-02-14T17:57:32.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:32 smithi184 ceph-mon[33536]: osdmap e696: 8 total, 8 up, 8 in 2024-02-14T17:57:33.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:33 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:33.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:33 smithi178 ceph-mon[39811]: osdmap e697: 8 total, 8 up, 8 in 2024-02-14T17:57:33.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:33.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:33 smithi178 ceph-mon[39811]: pgmap v1073: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:33.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:33 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:33.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:33 smithi178 ceph-mon[30321]: osdmap e697: 8 total, 8 up, 8 in 2024-02-14T17:57:33.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:33.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:33 smithi178 ceph-mon[30321]: pgmap v1073: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:33.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:33 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:33.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:33 smithi184 ceph-mon[33536]: osdmap e697: 8 total, 8 up, 8 in 2024-02-14T17:57:33.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:33.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:33 smithi184 ceph-mon[33536]: pgmap v1073: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:34.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:34.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:34 smithi178 ceph-mon[39811]: osdmap e698: 8 total, 8 up, 8 in 2024-02-14T17:57:34.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:34 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:34.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:34.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:34 smithi178 ceph-mon[30321]: osdmap e698: 8 total, 8 up, 8 in 2024-02-14T17:57:34.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:34 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:34 smithi184 ceph-mon[33536]: osdmap e698: 8 total, 8 up, 8 in 2024-02-14T17:57:34.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:34 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:35.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:35.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:35 smithi178 ceph-mon[39811]: osdmap e699: 8 total, 8 up, 8 in 2024-02-14T17:57:35.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:35 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:57:35.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:35 smithi178 ceph-mon[39811]: pgmap v1076: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:35.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:35.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:35 smithi178 ceph-mon[30321]: osdmap e699: 8 total, 8 up, 8 in 2024-02-14T17:57:35.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:35 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:57:35.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:35 smithi178 ceph-mon[30321]: pgmap v1076: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:35 smithi184 ceph-mon[33536]: osdmap e699: 8 total, 8 up, 8 in 2024-02-14T17:57:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:35 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:57:35.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:35 smithi184 ceph-mon[33536]: pgmap v1076: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:57:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:57:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:36.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_count","val": "3"}]': finished 2024-02-14T17:57:36.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:36 smithi178 ceph-mon[39811]: osdmap e700: 8 total, 8 up, 8 in 2024-02-14T17:57:36.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:36 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:57:36.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_count","val": "3"}]': finished 2024-02-14T17:57:36.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:36 smithi178 ceph-mon[30321]: osdmap e700: 8 total, 8 up, 8 in 2024-02-14T17:57:36.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:36 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:57:36.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_count","val": "3"}]': finished 2024-02-14T17:57:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:36 smithi184 ceph-mon[33536]: osdmap e700: 8 total, 8 up, 8 in 2024-02-14T17:57:36.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:36 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:57:37.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_period","val": "3"}]': finished 2024-02-14T17:57:37.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:37 smithi178 ceph-mon[39811]: osdmap e701: 8 total, 8 up, 8 in 2024-02-14T17:57:37.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:37.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:37 smithi178 ceph-mon[39811]: pgmap v1079: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:37.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:37 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:57:37.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_period","val": "3"}]': finished 2024-02-14T17:57:37.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:37 smithi178 ceph-mon[30321]: osdmap e701: 8 total, 8 up, 8 in 2024-02-14T17:57:37.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:37.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:37 smithi178 ceph-mon[30321]: pgmap v1079: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:37.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:37 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:57:37.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_period","val": "3"}]': finished 2024-02-14T17:57:37.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:37 smithi184 ceph-mon[33536]: osdmap e701: 8 total, 8 up, 8 in 2024-02-14T17:57:37.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:37.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:37 smithi184 ceph-mon[33536]: pgmap v1079: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:37.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:37 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:57:38.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:38 smithi178 ceph-mon[39811]: osdmap e702: 8 total, 8 up, 8 in 2024-02-14T17:57:38.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:57:38.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T17:57:38.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:38 smithi178 ceph-mon[39811]: osdmap e703: 8 total, 8 up, 8 in 2024-02-14T17:57:38.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:38 smithi178 ceph-mon[30321]: osdmap e702: 8 total, 8 up, 8 in 2024-02-14T17:57:38.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:57:38.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T17:57:38.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:38 smithi178 ceph-mon[30321]: osdmap e703: 8 total, 8 up, 8 in 2024-02-14T17:57:38.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:38 smithi184 ceph-mon[33536]: osdmap e702: 8 total, 8 up, 8 in 2024-02-14T17:57:38.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:57:38.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T17:57:38.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:38 smithi184 ceph-mon[33536]: osdmap e703: 8 total, 8 up, 8 in 2024-02-14T17:57:39.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:39 smithi178 ceph-mon[39811]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:39.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:39 smithi178 ceph-mon[30321]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:39.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:39 smithi184 ceph-mon[33536]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:40.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:40 smithi184 ceph-mon[33536]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-14T17:57:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:40 smithi178 ceph-mon[30321]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-14T17:57:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:40 smithi178 ceph-mon[39811]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-14T17:57:41.968 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:57:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:57:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:42 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:42.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:42 smithi184 ceph-mon[33536]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 836 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-02-14T17:57:42.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:42 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:42.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:42 smithi178 ceph-mon[30321]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 836 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-02-14T17:57:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:42 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:42 smithi178 ceph-mon[39811]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 836 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-02-14T17:57:43.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:57:43.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:43 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:43.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:43 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:43.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:57:43.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:43 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:43.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:43 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:57:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:43 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:43 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:44.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:57:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:57:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:44.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:44 smithi184 ceph-mon[33536]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-02-14T17:57:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:57:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:57:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:44.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:44 smithi178 ceph-mon[39811]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-02-14T17:57:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:57:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:57:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:44.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:44 smithi178 ceph-mon[30321]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-02-14T17:57:46.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:57:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:57:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:46.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:46 smithi184 ceph-mon[33536]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-02-14T17:57:46.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:46 smithi178 ceph-mon[39811]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-02-14T17:57:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:46 smithi178 ceph-mon[30321]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-02-14T17:57:48.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:48 smithi184 ceph-mon[33536]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1011 B/s rd, 6.5 KiB/s wr, 1 op/s 2024-02-14T17:57:48.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:48 smithi178 ceph-mon[30321]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1011 B/s rd, 6.5 KiB/s wr, 1 op/s 2024-02-14T17:57:48.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:48 smithi178 ceph-mon[39811]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1011 B/s rd, 6.5 KiB/s wr, 1 op/s 2024-02-14T17:57:50.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:50.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140"}]: dispatch 2024-02-14T17:57:50.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:50.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140"}]: dispatch 2024-02-14T17:57:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:57:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140"}]: dispatch 2024-02-14T17:57:51.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:57:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:57:51.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:51 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140"}]': finished 2024-02-14T17:57:51.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:51 smithi184 ceph-mon[33536]: osdmap e704: 8 total, 8 up, 8 in 2024-02-14T17:57:51.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:51 smithi184 ceph-mon[33536]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-02-14T17:57:51.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:51 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140"}]': finished 2024-02-14T17:57:51.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:51 smithi178 ceph-mon[30321]: osdmap e704: 8 total, 8 up, 8 in 2024-02-14T17:57:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:51 smithi178 ceph-mon[30321]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-02-14T17:57:51.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:51 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-140"}]': finished 2024-02-14T17:57:51.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:51 smithi178 ceph-mon[39811]: osdmap e704: 8 total, 8 up, 8 in 2024-02-14T17:57:51.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:51 smithi178 ceph-mon[39811]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-02-14T17:57:52.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:52 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:52.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:52 smithi184 ceph-mon[33536]: osdmap e705: 8 total, 8 up, 8 in 2024-02-14T17:57:52.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:52 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:52 smithi178 ceph-mon[39811]: osdmap e705: 8 total, 8 up, 8 in 2024-02-14T17:57:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:52 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:52 smithi178 ceph-mon[30321]: osdmap e705: 8 total, 8 up, 8 in 2024-02-14T17:57:53.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:53 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:53.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:53 smithi184 ceph-mon[33536]: pgmap v1092: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:57:53.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:53 smithi184 ceph-mon[33536]: osdmap e706: 8 total, 8 up, 8 in 2024-02-14T17:57:53.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:53 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:53.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:53 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:53.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:53 smithi178 ceph-mon[39811]: pgmap v1092: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:57:53.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:53 smithi178 ceph-mon[39811]: osdmap e706: 8 total, 8 up, 8 in 2024-02-14T17:57:53.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:53 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:53 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:53 smithi178 ceph-mon[30321]: pgmap v1092: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:57:53.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:53 smithi178 ceph-mon[30321]: osdmap e706: 8 total, 8 up, 8 in 2024-02-14T17:57:53.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:53 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:54.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:54.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:54 smithi184 ceph-mon[33536]: osdmap e707: 8 total, 8 up, 8 in 2024-02-14T17:57:54.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:54 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:54.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:54 smithi178 ceph-mon[39811]: osdmap e707: 8 total, 8 up, 8 in 2024-02-14T17:57:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:54 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:54 smithi178 ceph-mon[30321]: osdmap e707: 8 total, 8 up, 8 in 2024-02-14T17:57:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:54 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:55.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:55 smithi178 ceph-mon[39811]: pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:57:55.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:55.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:55 smithi178 ceph-mon[39811]: osdmap e708: 8 total, 8 up, 8 in 2024-02-14T17:57:55.689 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-142"}]: dispatch 2024-02-14T17:57:55.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:55 smithi178 ceph-mon[30321]: pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:57:55.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:55.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:55 smithi178 ceph-mon[30321]: osdmap e708: 8 total, 8 up, 8 in 2024-02-14T17:57:55.689 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-142"}]: dispatch 2024-02-14T17:57:55.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:55 smithi184 ceph-mon[33536]: pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 766 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:57:55.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:55.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:55 smithi184 ceph-mon[33536]: osdmap e708: 8 total, 8 up, 8 in 2024-02-14T17:57:55.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-142"}]: dispatch 2024-02-14T17:57:56.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:57:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:57:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-142"}]': finished 2024-02-14T17:57:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:56 smithi184 ceph-mon[33536]: osdmap e709: 8 total, 8 up, 8 in 2024-02-14T17:57:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-142", "mode": "writeback"}]: dispatch 2024-02-14T17:57:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:56 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-142", "mode": "writeback"}]': finished 2024-02-14T17:57:56.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-142"}]': finished 2024-02-14T17:57:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:56 smithi178 ceph-mon[30321]: osdmap e709: 8 total, 8 up, 8 in 2024-02-14T17:57:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-142", "mode": "writeback"}]: dispatch 2024-02-14T17:57:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:56 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:56.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-142", "mode": "writeback"}]': finished 2024-02-14T17:57:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-142"}]': finished 2024-02-14T17:57:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:56 smithi178 ceph-mon[39811]: osdmap e709: 8 total, 8 up, 8 in 2024-02-14T17:57:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-142", "mode": "writeback"}]: dispatch 2024-02-14T17:57:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:56 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-142", "mode": "writeback"}]': finished 2024-02-14T17:57:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:57 smithi184 ceph-mon[33536]: pgmap v1097: 204 pgs: 204 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:57:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:57 smithi184 ceph-mon[33536]: osdmap e710: 8 total, 8 up, 8 in 2024-02-14T17:57:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:57 smithi184 ceph-mon[33536]: osdmap e711: 8 total, 8 up, 8 in 2024-02-14T17:57:57.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:57.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[39811]: pgmap v1097: 204 pgs: 204 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:57:57.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[39811]: osdmap e710: 8 total, 8 up, 8 in 2024-02-14T17:57:57.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:57.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:57.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[39811]: osdmap e711: 8 total, 8 up, 8 in 2024-02-14T17:57:57.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[30321]: pgmap v1097: 204 pgs: 204 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:57:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[30321]: osdmap e710: 8 total, 8 up, 8 in 2024-02-14T17:57:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[30321]: osdmap e711: 8 total, 8 up, 8 in 2024-02-14T17:57:57.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:59.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:59 smithi184 ceph-mon[33536]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:57:59.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:59.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:59 smithi184 ceph-mon[33536]: osdmap e712: 8 total, 8 up, 8 in 2024-02-14T17:57:59.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:59.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:59 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:59.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:57:59 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[39811]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:57:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[39811]: osdmap e712: 8 total, 8 up, 8 in 2024-02-14T17:57:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[30321]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:57:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[30321]: osdmap e712: 8 total, 8 up, 8 in 2024-02-14T17:57:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:57:59.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:57:59 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:00 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:58:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:00 smithi184 ceph-mon[33536]: osdmap e713: 8 total, 8 up, 8 in 2024-02-14T17:58:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:58:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:58:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:00 smithi184 ceph-mon[33536]: osdmap e714: 8 total, 8 up, 8 in 2024-02-14T17:58:00.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:00 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:58:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:58:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[39811]: osdmap e713: 8 total, 8 up, 8 in 2024-02-14T17:58:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:58:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:58:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[39811]: osdmap e714: 8 total, 8 up, 8 in 2024-02-14T17:58:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:58:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:58:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[30321]: osdmap e713: 8 total, 8 up, 8 in 2024-02-14T17:58:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:58:00.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:58:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[30321]: osdmap e714: 8 total, 8 up, 8 in 2024-02-14T17:58:00.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:00 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:58:01.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:58:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:58:01.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:01 smithi184 ceph-mon[33536]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:01.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:01 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:01.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T17:58:01.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:01 smithi184 ceph-mon[33536]: osdmap e715: 8 total, 8 up, 8 in 2024-02-14T17:58:01.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:58:01.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:01 smithi178 ceph-mon[39811]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:01 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T17:58:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:01 smithi178 ceph-mon[39811]: osdmap e715: 8 total, 8 up, 8 in 2024-02-14T17:58:01.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:58:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:01 smithi178 ceph-mon[30321]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:01 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T17:58:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:01 smithi178 ceph-mon[30321]: osdmap e715: 8 total, 8 up, 8 in 2024-02-14T17:58:01.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:58:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:03 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:03 smithi184 ceph-mon[33536]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T17:58:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:03 smithi184 ceph-mon[33536]: osdmap e716: 8 total, 8 up, 8 in 2024-02-14T17:58:03.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:03.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:03 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:03.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:03 smithi178 ceph-mon[30321]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T17:58:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:03 smithi178 ceph-mon[30321]: osdmap e716: 8 total, 8 up, 8 in 2024-02-14T17:58:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:03 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:03 smithi178 ceph-mon[39811]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T17:58:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:03 smithi178 ceph-mon[39811]: osdmap e716: 8 total, 8 up, 8 in 2024-02-14T17:58:03.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:04.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:58:04.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:04 smithi184 ceph-mon[33536]: osdmap e717: 8 total, 8 up, 8 in 2024-02-14T17:58:04.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142"}]: dispatch 2024-02-14T17:58:04.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:04 smithi184 ceph-mon[33536]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:58:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:58:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:04 smithi178 ceph-mon[30321]: osdmap e717: 8 total, 8 up, 8 in 2024-02-14T17:58:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142"}]: dispatch 2024-02-14T17:58:04.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:04 smithi178 ceph-mon[30321]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:58:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:58:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:04 smithi178 ceph-mon[39811]: osdmap e717: 8 total, 8 up, 8 in 2024-02-14T17:58:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142"}]: dispatch 2024-02-14T17:58:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:04 smithi178 ceph-mon[39811]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:58:05.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142"}]': finished 2024-02-14T17:58:05.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:05 smithi178 ceph-mon[39811]: osdmap e718: 8 total, 8 up, 8 in 2024-02-14T17:58:05.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:05.688 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142"}]: dispatch 2024-02-14T17:58:05.688 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142"}]': finished 2024-02-14T17:58:05.688 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:05 smithi178 ceph-mon[30321]: osdmap e718: 8 total, 8 up, 8 in 2024-02-14T17:58:05.688 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:05.688 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142"}]: dispatch 2024-02-14T17:58:05.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142"}]': finished 2024-02-14T17:58:05.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:05 smithi184 ceph-mon[33536]: osdmap e718: 8 total, 8 up, 8 in 2024-02-14T17:58:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:05.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-142"}]: dispatch 2024-02-14T17:58:06.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:58:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:58:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:06 smithi184 ceph-mon[33536]: osdmap e719: 8 total, 8 up, 8 in 2024-02-14T17:58:06.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:06 smithi184 ceph-mon[33536]: pgmap v1112: 172 pgs: 172 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:58:06.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:06 smithi178 ceph-mon[39811]: osdmap e719: 8 total, 8 up, 8 in 2024-02-14T17:58:06.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:06 smithi178 ceph-mon[39811]: pgmap v1112: 172 pgs: 172 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:58:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:06 smithi178 ceph-mon[30321]: osdmap e719: 8 total, 8 up, 8 in 2024-02-14T17:58:06.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:06 smithi178 ceph-mon[30321]: pgmap v1112: 172 pgs: 172 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:58:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:07 smithi184 ceph-mon[33536]: osdmap e720: 8 total, 8 up, 8 in 2024-02-14T17:58:07.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:07.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:07 smithi178 ceph-mon[39811]: osdmap e720: 8 total, 8 up, 8 in 2024-02-14T17:58:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:07 smithi178 ceph-mon[30321]: osdmap e720: 8 total, 8 up, 8 in 2024-02-14T17:58:07.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:08 smithi184 ceph-mon[33536]: osdmap e721: 8 total, 8 up, 8 in 2024-02-14T17:58:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:08.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:08 smithi184 ceph-mon[33536]: pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:58:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:08 smithi178 ceph-mon[39811]: osdmap e721: 8 total, 8 up, 8 in 2024-02-14T17:58:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:08 smithi178 ceph-mon[39811]: pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:58:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:08 smithi178 ceph-mon[30321]: osdmap e721: 8 total, 8 up, 8 in 2024-02-14T17:58:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:08 smithi178 ceph-mon[30321]: pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:58:09.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:09.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:09 smithi184 ceph-mon[33536]: osdmap e722: 8 total, 8 up, 8 in 2024-02-14T17:58:09.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:09 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-144"}]: dispatch 2024-02-14T17:58:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:09 smithi178 ceph-mon[39811]: osdmap e722: 8 total, 8 up, 8 in 2024-02-14T17:58:09.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:09 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-144"}]: dispatch 2024-02-14T17:58:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:09 smithi178 ceph-mon[30321]: osdmap e722: 8 total, 8 up, 8 in 2024-02-14T17:58:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:09 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-144"}]: dispatch 2024-02-14T17:58:10.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-144"}]': finished 2024-02-14T17:58:10.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:10 smithi184 ceph-mon[33536]: osdmap e723: 8 total, 8 up, 8 in 2024-02-14T17:58:10.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-144", "mode": "readproxy"}]: dispatch 2024-02-14T17:58:10.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:10 smithi184 ceph-mon[33536]: pgmap v1118: 204 pgs: 30 unknown, 174 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:10.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:10 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:10.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:10 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-144", "mode": "readproxy"}]': finished 2024-02-14T17:58:10.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:10 smithi184 ceph-mon[33536]: osdmap e724: 8 total, 8 up, 8 in 2024-02-14T17:58:10.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-144"}]': finished 2024-02-14T17:58:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[39811]: osdmap e723: 8 total, 8 up, 8 in 2024-02-14T17:58:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-144", "mode": "readproxy"}]: dispatch 2024-02-14T17:58:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[39811]: pgmap v1118: 204 pgs: 30 unknown, 174 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-144", "mode": "readproxy"}]': finished 2024-02-14T17:58:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[39811]: osdmap e724: 8 total, 8 up, 8 in 2024-02-14T17:58:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-144"}]': finished 2024-02-14T17:58:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[30321]: osdmap e723: 8 total, 8 up, 8 in 2024-02-14T17:58:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-144", "mode": "readproxy"}]: dispatch 2024-02-14T17:58:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[30321]: pgmap v1118: 204 pgs: 30 unknown, 174 active+clean; 4.4 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-144", "mode": "readproxy"}]': finished 2024-02-14T17:58:10.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:10 smithi178 ceph-mon[30321]: osdmap e724: 8 total, 8 up, 8 in 2024-02-14T17:58:11.968 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:58:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:58:12.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:12 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:12 smithi178 ceph-mon[30321]: pgmap v1120: 204 pgs: 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 206 B/s wr, 1 op/s 2024-02-14T17:58:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:12 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:12 smithi178 ceph-mon[39811]: pgmap v1120: 204 pgs: 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 206 B/s wr, 1 op/s 2024-02-14T17:58:12.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:12 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:12 smithi184 ceph-mon[33536]: pgmap v1120: 204 pgs: 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 206 B/s wr, 1 op/s 2024-02-14T17:58:14.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:14.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:14.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:15.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:14 smithi178 ceph-mon[30321]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-02-14T17:58:15.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:14 smithi178 ceph-mon[39811]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-02-14T17:58:15.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:14 smithi184 ceph-mon[33536]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 773 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-02-14T17:58:15.983 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:58:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:58:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:16.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:16 smithi178 ceph-mon[39811]: pgmap v1122: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 128 B/s wr, 1 op/s 2024-02-14T17:58:16.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:16 smithi178 ceph-mon[30321]: pgmap v1122: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 128 B/s wr, 1 op/s 2024-02-14T17:58:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:16 smithi184 ceph-mon[33536]: pgmap v1122: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 128 B/s wr, 1 op/s 2024-02-14T17:58:18.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:18 smithi178 ceph-mon[39811]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 B/s wr, 1 op/s 2024-02-14T17:58:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:18 smithi178 ceph-mon[30321]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 B/s wr, 1 op/s 2024-02-14T17:58:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:18 smithi184 ceph-mon[33536]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 B/s wr, 1 op/s 2024-02-14T17:58:20.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:20 smithi178 ceph-mon[30321]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-14T17:58:20.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:20 smithi178 ceph-mon[39811]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-14T17:58:20.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:20 smithi184 ceph-mon[33536]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-14T17:58:21.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:21.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:21.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:21.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:58:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:58:22.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:58:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:22 smithi178 ceph-mon[39811]: osdmap e725: 8 total, 8 up, 8 in 2024-02-14T17:58:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144"}]: dispatch 2024-02-14T17:58:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:22 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:22 smithi178 ceph-mon[39811]: pgmap v1126: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:58:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:58:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:22 smithi178 ceph-mon[30321]: osdmap e725: 8 total, 8 up, 8 in 2024-02-14T17:58:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144"}]: dispatch 2024-02-14T17:58:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:22 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:22 smithi178 ceph-mon[30321]: pgmap v1126: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:58:22.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:58:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:22 smithi184 ceph-mon[33536]: osdmap e725: 8 total, 8 up, 8 in 2024-02-14T17:58:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144"}]: dispatch 2024-02-14T17:58:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:22 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:22 smithi184 ceph-mon[33536]: pgmap v1126: 204 pgs: 204 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:58:23.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:23 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:23.967 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144"}]': finished 2024-02-14T17:58:23.967 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:23 smithi178 ceph-mon[30321]: osdmap e726: 8 total, 8 up, 8 in 2024-02-14T17:58:23.967 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:23.967 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144"}]: dispatch 2024-02-14T17:58:23.967 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:23 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:23.967 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144"}]': finished 2024-02-14T17:58:23.968 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:23 smithi178 ceph-mon[39811]: osdmap e726: 8 total, 8 up, 8 in 2024-02-14T17:58:23.968 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:23.968 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144"}]: dispatch 2024-02-14T17:58:23.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:23 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:23.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144"}]': finished 2024-02-14T17:58:23.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:23 smithi184 ceph-mon[33536]: osdmap e726: 8 total, 8 up, 8 in 2024-02-14T17:58:23.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:23.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-144"}]: dispatch 2024-02-14T17:58:24.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:24 smithi178 ceph-mon[30321]: osdmap e727: 8 total, 8 up, 8 in 2024-02-14T17:58:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:24 smithi178 ceph-mon[30321]: pgmap v1129: 172 pgs: 172 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:58:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:24 smithi178 ceph-mon[39811]: osdmap e727: 8 total, 8 up, 8 in 2024-02-14T17:58:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:24 smithi178 ceph-mon[39811]: pgmap v1129: 172 pgs: 172 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:58:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:24 smithi184 ceph-mon[33536]: osdmap e727: 8 total, 8 up, 8 in 2024-02-14T17:58:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:24 smithi184 ceph-mon[33536]: pgmap v1129: 172 pgs: 172 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:58:25.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:58:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:58:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:25.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:25 smithi178 ceph-mon[39811]: osdmap e728: 8 total, 8 up, 8 in 2024-02-14T17:58:25.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:25 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:25.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:25 smithi178 ceph-mon[30321]: osdmap e728: 8 total, 8 up, 8 in 2024-02-14T17:58:25.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:25 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:25.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:25 smithi184 ceph-mon[33536]: osdmap e728: 8 total, 8 up, 8 in 2024-02-14T17:58:25.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:25 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:26.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:26.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:26 smithi178 ceph-mon[39811]: osdmap e729: 8 total, 8 up, 8 in 2024-02-14T17:58:26.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:26 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:26.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:26 smithi178 ceph-mon[39811]: pgmap v1132: 204 pgs: 9 creating+activating, 12 creating+peering, 183 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:26 smithi178 ceph-mon[30321]: osdmap e729: 8 total, 8 up, 8 in 2024-02-14T17:58:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:26 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:26 smithi178 ceph-mon[30321]: pgmap v1132: 204 pgs: 9 creating+activating, 12 creating+peering, 183 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:26.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:26.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:26 smithi184 ceph-mon[33536]: osdmap e729: 8 total, 8 up, 8 in 2024-02-14T17:58:26.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:26 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:26.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:26 smithi184 ceph-mon[33536]: pgmap v1132: 204 pgs: 9 creating+activating, 12 creating+peering, 183 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:27 smithi178 ceph-mon[30321]: osdmap e730: 8 total, 8 up, 8 in 2024-02-14T17:58:27.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:27 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-146"}]: dispatch 2024-02-14T17:58:27.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:27.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:27 smithi178 ceph-mon[39811]: osdmap e730: 8 total, 8 up, 8 in 2024-02-14T17:58:27.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:27 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-146"}]: dispatch 2024-02-14T17:58:27.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:27.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:27 smithi184 ceph-mon[33536]: osdmap e730: 8 total, 8 up, 8 in 2024-02-14T17:58:27.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:27 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-146"}]: dispatch 2024-02-14T17:58:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-146"}]': finished 2024-02-14T17:58:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[39811]: osdmap e731: 8 total, 8 up, 8 in 2024-02-14T17:58:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-146", "mode": "writeback"}]: dispatch 2024-02-14T17:58:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[39811]: pgmap v1135: 204 pgs: 9 creating+activating, 12 creating+peering, 183 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-146", "mode": "writeback"}]': finished 2024-02-14T17:58:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[39811]: osdmap e732: 8 total, 8 up, 8 in 2024-02-14T17:58:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-146"}]': finished 2024-02-14T17:58:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[30321]: osdmap e731: 8 total, 8 up, 8 in 2024-02-14T17:58:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-146", "mode": "writeback"}]: dispatch 2024-02-14T17:58:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[30321]: pgmap v1135: 204 pgs: 9 creating+activating, 12 creating+peering, 183 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-146", "mode": "writeback"}]': finished 2024-02-14T17:58:28.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:28 smithi178 ceph-mon[30321]: osdmap e732: 8 total, 8 up, 8 in 2024-02-14T17:58:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi178-80903-111", "overlaypool": "test-rados-api-smithi178-80903-146"}]': finished 2024-02-14T17:58:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:28 smithi184 ceph-mon[33536]: osdmap e731: 8 total, 8 up, 8 in 2024-02-14T17:58:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-146", "mode": "writeback"}]: dispatch 2024-02-14T17:58:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:28 smithi184 ceph-mon[33536]: pgmap v1135: 204 pgs: 9 creating+activating, 12 creating+peering, 183 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:28 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:28 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi178-80903-146", "mode": "writeback"}]': finished 2024-02-14T17:58:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:28 smithi184 ceph-mon[33536]: osdmap e732: 8 total, 8 up, 8 in 2024-02-14T17:58:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:29 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:58:29.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:29 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:29.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:29 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:58:29.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:29 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:29.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:29 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:58:29.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:29 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:30.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:58:30.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:30 smithi178 ceph-mon[39811]: osdmap e733: 8 total, 8 up, 8 in 2024-02-14T17:58:30.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:30 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:58:30.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:30 smithi178 ceph-mon[39811]: pgmap v1138: 204 pgs: 9 creating+activating, 10 creating+peering, 185 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:58:30.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:58:30.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:30 smithi178 ceph-mon[30321]: osdmap e733: 8 total, 8 up, 8 in 2024-02-14T17:58:30.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:30 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:58:30.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:30 smithi178 ceph-mon[30321]: pgmap v1138: 204 pgs: 9 creating+activating, 10 creating+peering, 185 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:58:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:58:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:30 smithi184 ceph-mon[33536]: osdmap e733: 8 total, 8 up, 8 in 2024-02-14T17:58:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:30 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:58:30.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:30 smithi184 ceph-mon[33536]: pgmap v1138: 204 pgs: 9 creating+activating, 10 creating+peering, 185 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:58:31.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:58:31.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[39811]: osdmap e734: 8 total, 8 up, 8 in 2024-02-14T17:58:31.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:58:31.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:31.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:58:31.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[39811]: osdmap e735: 8 total, 8 up, 8 in 2024-02-14T17:58:31.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:58:31.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[30321]: osdmap e734: 8 total, 8 up, 8 in 2024-02-14T17:58:31.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:58:31.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:31.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:58:31.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:31 smithi178 ceph-mon[30321]: osdmap e735: 8 total, 8 up, 8 in 2024-02-14T17:58:31.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:58:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:58:31.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:58:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:31 smithi184 ceph-mon[33536]: osdmap e734: 8 total, 8 up, 8 in 2024-02-14T17:58:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:58:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:31 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:31 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:58:31.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:31 smithi184 ceph-mon[33536]: osdmap e735: 8 total, 8 up, 8 in 2024-02-14T17:58:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:58:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:32 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:32 smithi178 ceph-mon[39811]: pgmap v1141: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-14T17:58:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:32 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:58:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:32 smithi178 ceph-mon[39811]: osdmap e736: 8 total, 8 up, 8 in 2024-02-14T17:58:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:58:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:32 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:32 smithi178 ceph-mon[30321]: pgmap v1141: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-14T17:58:32.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:32 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:58:32.834 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:32 smithi178 ceph-mon[30321]: osdmap e736: 8 total, 8 up, 8 in 2024-02-14T17:58:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:58:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:32 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:32 smithi184 ceph-mon[33536]: pgmap v1141: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-14T17:58:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:32 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:58:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:32 smithi184 ceph-mon[33536]: osdmap e736: 8 total, 8 up, 8 in 2024-02-14T17:58:33.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:58:33.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:33 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "target_max_objects","val": "1"}]': finished 2024-02-14T17:58:33.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:33 smithi178 ceph-mon[39811]: osdmap e737: 8 total, 8 up, 8 in 2024-02-14T17:58:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:58:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:33 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "target_max_objects","val": "1"}]': finished 2024-02-14T17:58:33.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:33 smithi178 ceph-mon[30321]: osdmap e737: 8 total, 8 up, 8 in 2024-02-14T17:58:33.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:58:33.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:33 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-146","var": "target_max_objects","val": "1"}]': finished 2024-02-14T17:58:33.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:33 smithi184 ceph-mon[33536]: osdmap e737: 8 total, 8 up, 8 in 2024-02-14T17:58:34.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:34 smithi178 ceph-mon[39811]: pgmap v1144: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 3 op/s 2024-02-14T17:58:34.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:34 smithi178 ceph-mon[30321]: pgmap v1144: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 3 op/s 2024-02-14T17:58:34.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:34 smithi184 ceph-mon[33536]: pgmap v1144: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 3 op/s 2024-02-14T17:58:35.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:35 smithi178 ceph-mon[39811]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T17:58:35.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:35 smithi178 ceph-mon[30321]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T17:58:35.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:58:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:58:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:35.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:35 smithi184 ceph-mon[33536]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T17:58:36.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:36 smithi178 ceph-mon[30321]: pgmap v1145: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 872 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:36.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:36 smithi178 ceph-mon[39811]: pgmap v1145: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 872 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:36.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:36 smithi184 ceph-mon[33536]: pgmap v1145: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 872 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:38.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:38 smithi178 ceph-mon[39811]: pgmap v1146: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:38 smithi178 ceph-mon[30321]: pgmap v1146: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:38.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:38 smithi184 ceph-mon[33536]: pgmap v1146: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:40.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:40 smithi178 ceph-mon[39811]: pgmap v1147: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:40 smithi178 ceph-mon[30321]: pgmap v1147: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:40.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:40 smithi184 ceph-mon[33536]: pgmap v1147: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:41.968 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:58:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:58:42.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:42 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:42.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:42 smithi178 ceph-mon[39811]: pgmap v1148: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:42 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:42 smithi178 ceph-mon[30321]: pgmap v1148: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:42.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:42 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:42.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:42 smithi184 ceph-mon[33536]: pgmap v1148: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:43.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:43.969 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 17:58:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a[66722]: logger=sqlstore.transactions t=2024-02-14T17:58:43.8519067Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-14T17:58:44.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:44.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:58:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:44 smithi184 ceph-mon[33536]: osdmap e738: 8 total, 8 up, 8 in 2024-02-14T17:58:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146"}]: dispatch 2024-02-14T17:58:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:58:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:44 smithi184 ceph-mon[33536]: pgmap v1150: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:44 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146"}]': finished 2024-02-14T17:58:44.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:44 smithi184 ceph-mon[33536]: osdmap e739: 8 total, 8 up, 8 in 2024-02-14T17:58:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:58:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[30321]: osdmap e738: 8 total, 8 up, 8 in 2024-02-14T17:58:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146"}]: dispatch 2024-02-14T17:58:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:58:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[30321]: pgmap v1150: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:45.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146"}]': finished 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[30321]: osdmap e739: 8 total, 8 up, 8 in 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[39811]: osdmap e738: 8 total, 8 up, 8 in 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146"}]: dispatch 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[39811]: pgmap v1150: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146"}]': finished 2024-02-14T17:58:45.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:44 smithi178 ceph-mon[39811]: osdmap e739: 8 total, 8 up, 8 in 2024-02-14T17:58:45.984 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:58:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:58:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:45 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146"}]: dispatch 2024-02-14T17:58:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:45 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:58:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:45 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:58:45.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:45 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:45 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146"}]: dispatch 2024-02-14T17:58:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:45 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:58:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:45 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:58:45.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:45 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:46.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:46.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:45 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-146"}]: dispatch 2024-02-14T17:58:46.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:45 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:58:46.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:45 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:58:46.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:45 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:47.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:46 smithi184 ceph-mon[33536]: osdmap e740: 8 total, 8 up, 8 in 2024-02-14T17:58:47.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:46 smithi184 ceph-mon[33536]: pgmap v1153: 172 pgs: 172 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-14T17:58:47.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:46 smithi184 ceph-mon[33536]: osdmap e741: 8 total, 8 up, 8 in 2024-02-14T17:58:47.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:46 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:47.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:46 smithi178 ceph-mon[39811]: osdmap e740: 8 total, 8 up, 8 in 2024-02-14T17:58:47.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:46 smithi178 ceph-mon[39811]: pgmap v1153: 172 pgs: 172 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-14T17:58:47.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:46 smithi178 ceph-mon[39811]: osdmap e741: 8 total, 8 up, 8 in 2024-02-14T17:58:47.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:46 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:47.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:46 smithi178 ceph-mon[30321]: osdmap e740: 8 total, 8 up, 8 in 2024-02-14T17:58:47.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:46 smithi178 ceph-mon[30321]: pgmap v1153: 172 pgs: 172 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-14T17:58:47.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:46 smithi178 ceph-mon[30321]: osdmap e741: 8 total, 8 up, 8 in 2024-02-14T17:58:47.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:46 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:48.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:47 smithi184 ceph-mon[33536]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T17:58:48.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:48.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:47 smithi184 ceph-mon[33536]: osdmap e742: 8 total, 8 up, 8 in 2024-02-14T17:58:48.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:47 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:48.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:47 smithi178 ceph-mon[30321]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T17:58:48.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:48.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:47 smithi178 ceph-mon[30321]: osdmap e742: 8 total, 8 up, 8 in 2024-02-14T17:58:48.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:47 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:48.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:47 smithi178 ceph-mon[39811]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T17:58:48.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:48.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:47 smithi178 ceph-mon[39811]: osdmap e742: 8 total, 8 up, 8 in 2024-02-14T17:58:48.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:47 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:49.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:48 smithi184 ceph-mon[33536]: pgmap v1156: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:58:49.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:49.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:48 smithi184 ceph-mon[33536]: osdmap e743: 8 total, 8 up, 8 in 2024-02-14T17:58:49.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:48 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148"}]: dispatch 2024-02-14T17:58:49.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:48 smithi178 ceph-mon[39811]: pgmap v1156: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:58:49.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:49.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:48 smithi178 ceph-mon[39811]: osdmap e743: 8 total, 8 up, 8 in 2024-02-14T17:58:49.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:48 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148"}]: dispatch 2024-02-14T17:58:49.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:48 smithi178 ceph-mon[30321]: pgmap v1156: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:58:49.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:49.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:48 smithi178 ceph-mon[30321]: osdmap e743: 8 total, 8 up, 8 in 2024-02-14T17:58:49.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:48 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148"}]: dispatch 2024-02-14T17:58:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148"}]': finished 2024-02-14T17:58:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:50 smithi184 ceph-mon[33536]: osdmap e744: 8 total, 8 up, 8 in 2024-02-14T17:58:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:50 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148"}]: dispatch 2024-02-14T17:58:50.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:50 smithi184 ceph-mon[33536]: pgmap v1159: 204 pgs: 29 unknown, 175 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148"}]': finished 2024-02-14T17:58:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:50 smithi178 ceph-mon[39811]: osdmap e744: 8 total, 8 up, 8 in 2024-02-14T17:58:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:50 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148"}]: dispatch 2024-02-14T17:58:51.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:50 smithi178 ceph-mon[39811]: pgmap v1159: 204 pgs: 29 unknown, 175 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:51.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148"}]': finished 2024-02-14T17:58:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:50 smithi178 ceph-mon[30321]: osdmap e744: 8 total, 8 up, 8 in 2024-02-14T17:58:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:50 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-148"}]: dispatch 2024-02-14T17:58:51.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:50 smithi178 ceph-mon[30321]: pgmap v1159: 204 pgs: 29 unknown, 175 active+clean; 4.4 MiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:51.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:58:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:58:51.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:51 smithi184 ceph-mon[33536]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:51.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:51 smithi184 ceph-mon[33536]: osdmap e745: 8 total, 8 up, 8 in 2024-02-14T17:58:52.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:51 smithi178 ceph-mon[39811]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:52.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:51 smithi178 ceph-mon[39811]: osdmap e745: 8 total, 8 up, 8 in 2024-02-14T17:58:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:51 smithi178 ceph-mon[30321]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:52.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:51 smithi178 ceph-mon[30321]: osdmap e745: 8 total, 8 up, 8 in 2024-02-14T17:58:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:52 smithi184 ceph-mon[33536]: osdmap e746: 8 total, 8 up, 8 in 2024-02-14T17:58:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:52 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:52 smithi184 ceph-mon[33536]: pgmap v1162: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:58:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:52 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:52 smithi184 ceph-mon[33536]: osdmap e747: 8 total, 8 up, 8 in 2024-02-14T17:58:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[39811]: osdmap e746: 8 total, 8 up, 8 in 2024-02-14T17:58:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[39811]: pgmap v1162: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:58:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:53.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[39811]: osdmap e747: 8 total, 8 up, 8 in 2024-02-14T17:58:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[30321]: osdmap e746: 8 total, 8 up, 8 in 2024-02-14T17:58:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:53.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[30321]: pgmap v1162: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:58:53.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:53.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:52 smithi178 ceph-mon[30321]: osdmap e747: 8 total, 8 up, 8 in 2024-02-14T17:58:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:53 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:53 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-150"}]: dispatch 2024-02-14T17:58:53.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:53 smithi184 ceph-mon[33536]: osdmap e748: 8 total, 8 up, 8 in 2024-02-14T17:58:54.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:53 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:54.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:53 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-150"}]: dispatch 2024-02-14T17:58:54.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:53 smithi178 ceph-mon[39811]: osdmap e748: 8 total, 8 up, 8 in 2024-02-14T17:58:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:53 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:53 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-150"}]: dispatch 2024-02-14T17:58:54.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:53 smithi178 ceph-mon[30321]: osdmap e748: 8 total, 8 up, 8 in 2024-02-14T17:58:54.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:54 smithi184 ceph-mon[33536]: pgmap v1165: 172 pgs: 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-02-14T17:58:55.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:54 smithi178 ceph-mon[39811]: pgmap v1165: 172 pgs: 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-02-14T17:58:55.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:54 smithi178 ceph-mon[30321]: pgmap v1165: 172 pgs: 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-02-14T17:58:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:55 smithi184 ceph-mon[33536]: osdmap e749: 8 total, 8 up, 8 in 2024-02-14T17:58:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:55 smithi178 ceph-mon[39811]: osdmap e749: 8 total, 8 up, 8 in 2024-02-14T17:58:56.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:56.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:58:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:58:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:56.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:55 smithi178 ceph-mon[30321]: osdmap e749: 8 total, 8 up, 8 in 2024-02-14T17:58:56.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:56 smithi184 ceph-mon[33536]: osdmap e750: 8 total, 8 up, 8 in 2024-02-14T17:58:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:56 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-152"}]: dispatch 2024-02-14T17:58:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:56 smithi184 ceph-mon[33536]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:56.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:56 smithi184 ceph-mon[33536]: pgmap v1168: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:58:57.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:57.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[39811]: osdmap e750: 8 total, 8 up, 8 in 2024-02-14T17:58:57.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-152"}]: dispatch 2024-02-14T17:58:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[39811]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:57.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[39811]: pgmap v1168: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:58:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[30321]: osdmap e750: 8 total, 8 up, 8 in 2024-02-14T17:58:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-152"}]: dispatch 2024-02-14T17:58:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[30321]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:57.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:56 smithi178 ceph-mon[30321]: pgmap v1168: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:58:57.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:57 smithi184 ceph-mon[33536]: osdmap e751: 8 total, 8 up, 8 in 2024-02-14T17:58:58.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:57 smithi178 ceph-mon[39811]: osdmap e751: 8 total, 8 up, 8 in 2024-02-14T17:58:58.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:57 smithi178 ceph-mon[30321]: osdmap e751: 8 total, 8 up, 8 in 2024-02-14T17:58:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:58 smithi184 ceph-mon[33536]: osdmap e752: 8 total, 8 up, 8 in 2024-02-14T17:58:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:58.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:58 smithi184 ceph-mon[33536]: pgmap v1171: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:58:59.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:58 smithi178 ceph-mon[39811]: osdmap e752: 8 total, 8 up, 8 in 2024-02-14T17:58:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:59.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:58 smithi178 ceph-mon[39811]: pgmap v1171: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:58:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:58 smithi178 ceph-mon[30321]: osdmap e752: 8 total, 8 up, 8 in 2024-02-14T17:58:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:59.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:58 smithi178 ceph-mon[30321]: pgmap v1171: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:58:59.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:59 smithi184 ceph-mon[33536]: osdmap e753: 8 total, 8 up, 8 in 2024-02-14T17:58:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-111","var": "dedup_tier","val": "test-rados-api-smithi178-80903-154"}]: dispatch 2024-02-14T17:58:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:58:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:59 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-154"}]: dispatch 2024-02-14T17:58:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:59 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:58:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:58:59 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[39811]: osdmap e753: 8 total, 8 up, 8 in 2024-02-14T17:59:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-111","var": "dedup_tier","val": "test-rados-api-smithi178-80903-154"}]: dispatch 2024-02-14T17:59:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:59:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-154"}]: dispatch 2024-02-14T17:59:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:59:00.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:00.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi178-80903-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:00.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[30321]: osdmap e753: 8 total, 8 up, 8 in 2024-02-14T17:59:00.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi178-80903-111","var": "dedup_tier","val": "test-rados-api-smithi178-80903-154"}]: dispatch 2024-02-14T17:59:00.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:59:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi178-80903-111", "tierpool": "test-rados-api-smithi178-80903-154"}]: dispatch 2024-02-14T17:59:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:59:00.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:58:59 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:00.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:00 smithi184 ceph-mon[33536]: osdmap e754: 8 total, 8 up, 8 in 2024-02-14T17:59:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:00 smithi184 ceph-mon[33536]: pgmap v1174: 172 pgs: 172 active+clean; 4.4 MiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:01.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:00 smithi178 ceph-mon[39811]: osdmap e754: 8 total, 8 up, 8 in 2024-02-14T17:59:01.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:00 smithi178 ceph-mon[39811]: pgmap v1174: 172 pgs: 172 active+clean; 4.4 MiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:00 smithi178 ceph-mon[30321]: osdmap e754: 8 total, 8 up, 8 in 2024-02-14T17:59:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:00 smithi178 ceph-mon[30321]: pgmap v1174: 172 pgs: 172 active+clean; 4.4 MiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:01.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:59:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:59:01.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:01 smithi184 ceph-mon[33536]: osdmap e755: 8 total, 8 up, 8 in 2024-02-14T17:59:01.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:01 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:59:02.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:01 smithi178 ceph-mon[39811]: osdmap e755: 8 total, 8 up, 8 in 2024-02-14T17:59:02.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:01 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:59:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:01 smithi178 ceph-mon[30321]: osdmap e755: 8 total, 8 up, 8 in 2024-02-14T17:59:02.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:01 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:59:02.717 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7889 ms) 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12575 ms) 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8051 ms) 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13077 ms) 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7315 ms) 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8050 ms) 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: ok, hit_set contains 337:602f83fe:::foo:head 2024-02-14T17:59:02.718 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (8105 ms) 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: first is 1707933459 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,1707933461,1707933462,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,1707933461,1707933462,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,1707933461,1707933462,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,1707933461,1707933462,1707933464,1707933465,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,1707933461,1707933462,1707933464,1707933465,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,1707933461,1707933462,1707933464,1707933465,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,1707933461,1707933462,1707933464,1707933465,1707933467,1707933468,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,1707933461,1707933462,1707933464,1707933465,1707933467,1707933468,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933459,1707933461,1707933462,1707933464,1707933465,1707933467,1707933468,0 2024-02-14T17:59:02.719 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: got ls 1707933462,1707933464,1707933465,1707933467,1707933468,1707933470,1707933471,0 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: first now 1707933462, trimmed 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20145 ms) 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: foo0 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14053 ms) 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18057 ms) 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22126 ms) 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5014 ms) 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3021 ms) 2024-02-14T17:59:02.720 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-02-14T17:59:02.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3026 ms) 2024-02-14T17:59:02.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-02-14T17:59:02.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3015 ms) 2024-02-14T17:59:02.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (228955 ms total) 2024-02-14T17:59:02.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: 2024-02-14T17:59:02.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-02-14T17:59:02.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (858557 ms total) 2024-02-14T17:59:02.721 INFO:tasks.workunit.client.0.smithi178.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80621 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80621 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80834 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80834 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81373 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81373 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81136 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81136 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.726 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81015 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81015 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80705 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80705 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81037 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81037 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81596 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81596 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81682 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81682 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80722 2024-02-14T17:59:02.727 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80722 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81435 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81435 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81281 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81281 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80595 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80595 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80659 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80659 2024-02-14T17:59:02.728 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81314 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81314 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80586 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80586 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80606 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80606 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81778 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81778 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80892 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80892 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.729 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81864 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81864 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81024 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81024 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81515 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81515 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81917 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81917 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80964 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80964 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81335 2024-02-14T17:59:02.730 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81335 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81726 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81726 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81484 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81484 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81831 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81831 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81229 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81229 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80779 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80779 2024-02-14T17:59:02.731 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81874 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81874 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81151 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81151 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80579 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80579 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80639 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80639 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81049 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81049 2024-02-14T17:59:02.732 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=80680 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 80680 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81179 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81179 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81201 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81201 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81719 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81719 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=81941 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 81941 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ exit 0 2024-02-14T17:59:02.733 INFO:tasks.workunit.client.0.smithi178.stderr:+ cleanup 2024-02-14T17:59:02.734 INFO:tasks.workunit.client.0.smithi178.stderr:+ pkill -P 80574 2024-02-14T17:59:02.748 INFO:tasks.workunit.client.0.smithi178.stderr:+ true 2024-02-14T17:59:02.749 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-14T17:59:02.749 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-14T17:59:02.782 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-02-14T17:59:02.783 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-02-14T17:59:02.783 DEBUG:teuthology.orchestra.run.smithi178:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8727ce939fc044a98f215e29696bb1433f5758d1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-02-14T17:59:02.843 INFO:tasks.workunit.client.0.smithi178.stderr:++ uuidgen 2024-02-14T17:59:02.848 INFO:tasks.workunit.client.0.smithi178.stderr:+ p=c8dfcd28-16bd-46b5-a2ee-f0743e289c3b 2024-02-14T17:59:02.848 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool create c8dfcd28-16bd-46b5-a2ee-f0743e289c3b 12 2024-02-14T17:59:02.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.931+0000 7f66bff39640 1 -- 172.21.15.178:0/1939095464 >> v1:172.21.15.178:6789/0 conn(0x7f66b8100280 legacy=0x7f66b8100660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:02.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.931+0000 7f66bff39640 1 -- 172.21.15.178:0/1939095464 shutdown_connections 2024-02-14T17:59:02.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.931+0000 7f66bff39640 1 -- 172.21.15.178:0/1939095464 >> 172.21.15.178:0/1939095464 conn(0x7f66b8075a30 msgr2=0x7f66b8075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:59:02.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.931+0000 7f66bff39640 1 -- 172.21.15.178:0/1939095464 shutdown_connections 2024-02-14T17:59:02.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.931+0000 7f66bff39640 1 -- 172.21.15.178:0/1939095464 wait complete. 2024-02-14T17:59:02.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.932+0000 7f66bff39640 1 Processor -- start 2024-02-14T17:59:02.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.932+0000 7f66bff39640 1 -- start start 2024-02-14T17:59:02.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.932+0000 7f66bff39640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f66b81a5ea0 con 0x7f66b8100280 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.932+0000 7f66bff39640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f66b81a70a0 con 0x7f66b8108dc0 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.932+0000 7f66bff39640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f66b81a82a0 con 0x7f66b810c930 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.932+0000 7f66bdcd5640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f66b8100280 0x7f66b8068b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50238/0 (socket says 172.21.15.178:50238) 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.932+0000 7f66bdcd5640 1 -- 172.21.15.178:0/3335923010 learned_addr learned my addr 172.21.15.178:0/3335923010 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.932+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035653426 0 0) 0x7f66b81a5ea0 con 0x7f66b8100280 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.932+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66a0003520 con 0x7f66b8100280 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.932+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327772575 0 0) 0x7f66b81a82a0 con 0x7f66b810c930 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.932+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66b81a5ea0 con 0x7f66b810c930 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1772679883 0 0) 0x7f66b81a5ea0 con 0x7f66b810c930 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f66b81a82a0 con 0x7f66b810c930 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f66b00017f0 con 0x7f66b810c930 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3135482492 0 0) 0x7f66a0003520 con 0x7f66b8100280 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f66b81a5ea0 con 0x7f66b8100280 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f66a8001880 con 0x7f66b8100280 2024-02-14T17:59:02.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221798101 0 0) 0x7f66b81a70a0 con 0x7f66b8108dc0 2024-02-14T17:59:02.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66a0003520 con 0x7f66b8108dc0 2024-02-14T17:59:02.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2256529826 0 0) 0x7f66b81a82a0 con 0x7f66b810c930 2024-02-14T17:59:02.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 >> v1:172.21.15.184:6789/0 conn(0x7f66b8108dc0 legacy=0x7f66b81a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:02.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 >> v1:172.21.15.178:6789/0 conn(0x7f66b8100280 legacy=0x7f66b8068b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:02.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b81a94a0 con 0x7f66b810c930 2024-02-14T17:59:02.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66bff39640 1 -- 172.21.15.178:0/3335923010 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b81a8490 con 0x7f66b810c930 2024-02-14T17:59:02.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66bff39640 1 -- 172.21.15.178:0/3335923010 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b81a8920 con 0x7f66b810c930 2024-02-14T17:59:02.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f66b0003aa0 con 0x7f66b810c930 2024-02-14T17:59:02.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.933+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f66b00049d0 con 0x7f66b810c930 2024-02-14T17:59:02.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.934+0000 7f66bff39640 1 -- 172.21.15.178:0/3335923010 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a4005080 con 0x7f66b810c930 2024-02-14T17:59:02.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.934+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f66b00035f0 con 0x7f66b810c930 2024-02-14T17:59:02.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.935+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(757..757 src has 255..757) v4 ==== 7442+0+0 (unknown 2262768171 0 0) 0x7f66b0087bc0 con 0x7f66b810c930 2024-02-14T17:59:02.940 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:02.939+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f66b00549c0 con 0x7f66b810c930 2024-02-14T17:59:02.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:02 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:59:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:02 smithi184 ceph-mon[33536]: osdmap e756: 8 total, 8 up, 8 in 2024-02-14T17:59:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:02 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:59:02.970 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:02 smithi184 ceph-mon[33536]: pgmap v1177: 164 pgs: 164 active+clean; 582 KiB data, 824 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:59:03.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:02 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:59:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:02 smithi178 ceph-mon[39811]: osdmap e756: 8 total, 8 up, 8 in 2024-02-14T17:59:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:02 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:59:03.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:02 smithi178 ceph-mon[39811]: pgmap v1177: 164 pgs: 164 active+clean; 582 KiB data, 824 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:59:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:02 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:59:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:02 smithi178 ceph-mon[30321]: osdmap e756: 8 total, 8 up, 8 in 2024-02-14T17:59:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:02 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-111"}]: dispatch 2024-02-14T17:59:03.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:02 smithi178 ceph-mon[30321]: pgmap v1177: 164 pgs: 164 active+clean; 582 KiB data, 824 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:59:03.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.118+0000 7f66bff39640 1 -- 172.21.15.178:0/3335923010 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12} v 0) v1 -- 0x7f66a40058b0 con 0x7f66b810c930 2024-02-14T17:59:03.711 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.709+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.2 v1:172.21.15.178:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]=0 pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' created v758) v1 ==== 176+0+0 (unknown 4058672068 0 0) 0x7f66b0052d30 con 0x7f66b810c930 2024-02-14T17:59:03.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.818+0000 7f66bff39640 1 -- 172.21.15.178:0/3335923010 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12} v 0) v1 -- 0x7f66a40053e0 con 0x7f66b810c930 2024-02-14T17:59:03.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.819+0000 7f66b6ffd640 1 -- 172.21.15.178:0/3335923010 <== mon.2 v1:172.21.15.178:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]=0 pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' already exists v758) v1 ==== 183+0+0 (unknown 1500782082 0 0) 0x7f66a40053e0 con 0x7f66b810c930 2024-02-14T17:59:03.820 INFO:tasks.workunit.client.0.smithi178.stderr:pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' already exists 2024-02-14T17:59:03.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.821+0000 7f66bff39640 1 -- 172.21.15.178:0/3335923010 >> v1:172.21.15.178:6800/4205070280 conn(0x7f66a006e920 legacy=0x7f66a0070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:03.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.821+0000 7f66bff39640 1 -- 172.21.15.178:0/3335923010 >> v1:172.21.15.178:6790/0 conn(0x7f66b810c930 legacy=0x7f66b81a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:03.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.821+0000 7f66bff39640 1 -- 172.21.15.178:0/3335923010 shutdown_connections 2024-02-14T17:59:03.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.821+0000 7f66bff39640 1 -- 172.21.15.178:0/3335923010 >> 172.21.15.178:0/3335923010 conn(0x7f66b8075a30 msgr2=0x7f66b8110030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:59:03.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.821+0000 7f66bff39640 1 -- 172.21.15.178:0/3335923010 shutdown_connections 2024-02-14T17:59:03.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.821+0000 7f66bff39640 1 -- 172.21.15.178:0/3335923010 wait complete. 2024-02-14T17:59:03.831 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool set-quota c8dfcd28-16bd-46b5-a2ee-f0743e289c3b max_objects 10 2024-02-14T17:59:03.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.911+0000 7ff2e5265640 1 -- 172.21.15.178:0/1753666349 >> v1:172.21.15.178:6789/0 conn(0x7ff2e0109ed0 legacy=0x7ff2e010c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.911+0000 7ff2e5265640 1 -- 172.21.15.178:0/1753666349 shutdown_connections 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.911+0000 7ff2e5265640 1 -- 172.21.15.178:0/1753666349 >> 172.21.15.178:0/1753666349 conn(0x7ff2e00fcaf0 msgr2=0x7ff2e00fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.911+0000 7ff2e5265640 1 -- 172.21.15.178:0/1753666349 shutdown_connections 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.911+0000 7ff2e5265640 1 -- 172.21.15.178:0/1753666349 wait complete. 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.911+0000 7ff2e5265640 1 Processor -- start 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2e5265640 1 -- start start 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2e5265640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff2e01a5e80 con 0x7ff2e0101250 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2e5265640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff2e01a7080 con 0x7ff2e0109ed0 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2e5265640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff2e01a8280 con 0x7ff2e010da40 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2df7fe640 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7ff2e010da40 0x7ff2e01a4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:60902/0 (socket says 172.21.15.178:60902) 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2deffd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff2e0101250 0x7ff2e01064a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:42132/0 (socket says 172.21.15.178:42132) 2024-02-14T17:59:03.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2df7fe640 1 -- 172.21.15.178:0/2220241538 learned_addr learned my addr 172.21.15.178:0/2220241538 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88124261 0 0) 0x7ff2e01a8280 con 0x7ff2e010da40 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c8003520 con 0x7ff2e010da40 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077208673 0 0) 0x7ff2e01a5e80 con 0x7ff2e0101250 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e01a8280 con 0x7ff2e0101250 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267459417 0 0) 0x7ff2e01a7080 con 0x7ff2e0109ed0 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e01a5e80 con 0x7ff2e0109ed0 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2845939136 0 0) 0x7ff2c8003520 con 0x7ff2e010da40 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff2e01a7080 con 0x7ff2e010da40 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ff2d8001970 con 0x7ff2e010da40 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3179276107 0 0) 0x7ff2e01a8280 con 0x7ff2e0101250 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff2c8003520 con 0x7ff2e0101250 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ff2d0001b30 con 0x7ff2e0101250 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1753731494 0 0) 0x7ff2e01a5e80 con 0x7ff2e0109ed0 2024-02-14T17:59:03.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff2e01a8280 con 0x7ff2e0109ed0 2024-02-14T17:59:03.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.912+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ff2d4001510 con 0x7ff2e0109ed0 2024-02-14T17:59:03.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.913+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1722502983 0 0) 0x7ff2e01a7080 con 0x7ff2e010da40 2024-02-14T17:59:03.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.913+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 >> v1:172.21.15.184:6789/0 conn(0x7ff2e0109ed0 legacy=0x7ff2e01a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:03.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.913+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 >> v1:172.21.15.178:6789/0 conn(0x7ff2e0101250 legacy=0x7ff2e01064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:03.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.913+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e01a9480 con 0x7ff2e010da40 2024-02-14T17:59:03.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.913+0000 7ff2e5265640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e01a8470 con 0x7ff2e010da40 2024-02-14T17:59:03.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.913+0000 7ff2e5265640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e01a8900 con 0x7ff2e010da40 2024-02-14T17:59:03.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.913+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7ff2d8003a80 con 0x7ff2e010da40 2024-02-14T17:59:03.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.913+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7ff2d8005b60 con 0x7ff2e010da40 2024-02-14T17:59:03.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.914+0000 7ff2e5265640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2cc005080 con 0x7ff2e010da40 2024-02-14T17:59:03.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.914+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7ff2d80035f0 con 0x7ff2e010da40 2024-02-14T17:59:03.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.915+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(758..758 src has 255..758) v4 ==== 7817+0+0 (unknown 2683074579 0 0) 0x7ff2d8088f10 con 0x7ff2e010da40 2024-02-14T17:59:03.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:03.919+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7ff2d8055970 con 0x7ff2e010da40 2024-02-14T17:59:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:03 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:59:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:03 smithi184 ceph-mon[33536]: osdmap e757: 8 total, 8 up, 8 in 2024-02-14T17:59:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:03 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3335923010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:03.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:03 smithi184 ceph-mon[33536]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:03 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:59:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:03 smithi178 ceph-mon[30321]: osdmap e757: 8 total, 8 up, 8 in 2024-02-14T17:59:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:03 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3335923010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:04.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:03 smithi178 ceph-mon[30321]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:03 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/704810028' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi178-80903-111"}]': finished 2024-02-14T17:59:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:03 smithi178 ceph-mon[39811]: osdmap e757: 8 total, 8 up, 8 in 2024-02-14T17:59:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:03 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3335923010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:04.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:03 smithi178 ceph-mon[39811]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:04.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:04.094+0000 7ff2e5265640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7ff2cc005330 con 0x7ff2e010da40 2024-02-14T17:59:04.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:04.711+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.2 v1:172.21.15.178:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v759) v1 ==== 223+0+0 (unknown 825452765 0 0) 0x7ff2d8053f70 con 0x7ff2e010da40 2024-02-14T17:59:04.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:04.825+0000 7ff2e5265640 1 -- 172.21.15.178:0/2220241538 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7ff2cc001090 con 0x7ff2e010da40 2024-02-14T17:59:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[39811]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]': finished 2024-02-14T17:59:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[39811]: osdmap e758: 8 total, 8 up, 8 in 2024-02-14T17:59:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3335923010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[39811]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2220241538' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[39811]: from='client.56975 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:05.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[39811]: pgmap v1180: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 824 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:59:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[30321]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]': finished 2024-02-14T17:59:05.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[30321]: osdmap e758: 8 total, 8 up, 8 in 2024-02-14T17:59:05.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3335923010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:05.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[30321]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:05.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2220241538' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:05.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[30321]: from='client.56975 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:05.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:04 smithi178 ceph-mon[30321]: pgmap v1180: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 824 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:59:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:04 smithi184 ceph-mon[33536]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]': finished 2024-02-14T17:59:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:04 smithi184 ceph-mon[33536]: osdmap e758: 8 total, 8 up, 8 in 2024-02-14T17:59:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3335923010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:04 smithi184 ceph-mon[33536]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pg_num": 12}]: dispatch 2024-02-14T17:59:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:04 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2220241538' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:04 smithi184 ceph-mon[33536]: from='client.56975 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:05.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:04 smithi184 ceph-mon[33536]: pgmap v1180: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 824 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:59:05.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.722+0000 7ff2c7fff640 1 -- 172.21.15.178:0/2220241538 <== mon.2 v1:172.21.15.178:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v760) v1 ==== 223+0+0 (unknown 2347934238 0 0) 0x7ff2d8055790 con 0x7ff2e010da40 2024-02-14T17:59:05.724 INFO:tasks.workunit.client.0.smithi178.stderr:set-quota max_objects = 10 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b 2024-02-14T17:59:05.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.725+0000 7ff2e5265640 1 -- 172.21.15.178:0/2220241538 >> v1:172.21.15.178:6800/4205070280 conn(0x7ff2c806e9e0 legacy=0x7ff2c8070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:05.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.725+0000 7ff2e5265640 1 -- 172.21.15.178:0/2220241538 >> v1:172.21.15.178:6790/0 conn(0x7ff2e010da40 legacy=0x7ff2e01a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:05.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.725+0000 7ff2e5265640 1 -- 172.21.15.178:0/2220241538 shutdown_connections 2024-02-14T17:59:05.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.725+0000 7ff2e5265640 1 -- 172.21.15.178:0/2220241538 >> 172.21.15.178:0/2220241538 conn(0x7ff2e00fcaf0 msgr2=0x7ff2e01106d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:59:05.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.726+0000 7ff2e5265640 1 -- 172.21.15.178:0/2220241538 shutdown_connections 2024-02-14T17:59:05.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.726+0000 7ff2e5265640 1 -- 172.21.15.178:0/2220241538 wait complete. 2024-02-14T17:59:05.737 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool application enable c8dfcd28-16bd-46b5-a2ee-f0743e289c3b rados 2024-02-14T17:59:05.824 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.818+0000 7fe54b8f6640 1 -- 172.21.15.178:0/4237151476 >> v1:172.21.15.178:6790/0 conn(0x7fe544105f50 legacy=0x7fe5441039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:05.824 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe54b8f6640 1 -- 172.21.15.178:0/4237151476 shutdown_connections 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe54b8f6640 1 -- 172.21.15.178:0/4237151476 >> 172.21.15.178:0/4237151476 conn(0x7fe544075a30 msgr2=0x7fe544075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe54b8f6640 1 -- 172.21.15.178:0/4237151476 shutdown_connections 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe54b8f6640 1 -- 172.21.15.178:0/4237151476 wait complete. 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe54b8f6640 1 Processor -- start 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe54b8f6640 1 -- start start 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe54b8f6640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe5441ac310 con 0x7fe544105f50 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe54b8f6640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe5441ad4f0 con 0x7fe5441040e0 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe54b8f6640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe5441ae6f0 con 0x7fe544104570 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe549e93640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe544105f50 0x7fe5441aaad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:42136/0 (socket says 172.21.15.178:42136) 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe548e91640 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7fe544104570 0x7fe5441a73a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:60930/0 (socket says 172.21.15.178:60930) 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.819+0000 7fe549e93640 1 -- 172.21.15.178:0/2950442115 learned_addr learned my addr 172.21.15.178:0/2950442115 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532746792 0 0) 0x7fe5441ac310 con 0x7fe544105f50 2024-02-14T17:59:05.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe530003160 con 0x7fe544105f50 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623631421 0 0) 0x7fe5441ae6f0 con 0x7fe544104570 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5441ac310 con 0x7fe544104570 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1520729893 0 0) 0x7fe530003160 con 0x7fe544105f50 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe5441ae6f0 con 0x7fe544105f50 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fe53c0016d0 con 0x7fe544105f50 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 675901579 0 0) 0x7fe5441ac310 con 0x7fe544104570 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe530003160 con 0x7fe544104570 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732017550 0 0) 0x7fe5441ad4f0 con 0x7fe5441040e0 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5441ac310 con 0x7fe5441040e0 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fe5340017e0 con 0x7fe544104570 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3188367897 0 0) 0x7fe5441ae6f0 con 0x7fe544105f50 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 >> v1:172.21.15.178:6790/0 conn(0x7fe544104570 legacy=0x7fe5441a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:05.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 >> v1:172.21.15.184:6789/0 conn(0x7fe5441040e0 legacy=0x7fe544116290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:05.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5441af8f0 con 0x7fe544105f50 2024-02-14T17:59:05.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe54b8f6640 1 -- 172.21.15.178:0/2950442115 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5441ae8e0 con 0x7fe544105f50 2024-02-14T17:59:05.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe54b8f6640 1 -- 172.21.15.178:0/2950442115 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5441aedf0 con 0x7fe544105f50 2024-02-14T17:59:05.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fe53c001c60 con 0x7fe544105f50 2024-02-14T17:59:05.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.820+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fe53c005c80 con 0x7fe544105f50 2024-02-14T17:59:05.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.821+0000 7fe54b8f6640 1 -- 172.21.15.178:0/2950442115 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe528005080 con 0x7fe544105f50 2024-02-14T17:59:05.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.822+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7fe53c0038a0 con 0x7fe544105f50 2024-02-14T17:59:05.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.822+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(760..760 src has 255..760) v4 ==== 7817+0+0 (unknown 2321596689 0 0) 0x7fe53c088f00 con 0x7fe544105f50 2024-02-14T17:59:05.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:05.827+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fe53c055bf0 con 0x7fe544105f50 2024-02-14T17:59:06.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:06.000+0000 7fe54b8f6640 1 -- 172.21.15.178:0/2950442115 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"} v 0) v1 -- 0x7fe528005330 con 0x7fe544105f50 2024-02-14T17:59:06.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:59:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:06.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[39811]: from='client.56975 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]': finished 2024-02-14T17:59:06.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[39811]: osdmap e759: 8 total, 8 up, 8 in 2024-02-14T17:59:06.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2220241538' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:06.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[39811]: from='client.56975 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:06.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[39811]: from='client.56975 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]': finished 2024-02-14T17:59:06.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[39811]: osdmap e760: 8 total, 8 up, 8 in 2024-02-14T17:59:06.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[30321]: from='client.56975 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]': finished 2024-02-14T17:59:06.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[30321]: osdmap e759: 8 total, 8 up, 8 in 2024-02-14T17:59:06.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2220241538' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:06.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[30321]: from='client.56975 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:06.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[30321]: from='client.56975 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]': finished 2024-02-14T17:59:06.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:05 smithi178 ceph-mon[30321]: osdmap e760: 8 total, 8 up, 8 in 2024-02-14T17:59:06.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:05 smithi184 ceph-mon[33536]: from='client.56975 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]': finished 2024-02-14T17:59:06.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:05 smithi184 ceph-mon[33536]: osdmap e759: 8 total, 8 up, 8 in 2024-02-14T17:59:06.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:05 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2220241538' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:06.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:05 smithi184 ceph-mon[33536]: from='client.56975 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T17:59:06.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:05 smithi184 ceph-mon[33536]: from='client.56975 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "10"}]': finished 2024-02-14T17:59:06.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:05 smithi184 ceph-mon[33536]: osdmap e760: 8 total, 8 up, 8 in 2024-02-14T17:59:06.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:06.731+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]=0 enabled application 'rados' on pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' v761) v1 ==== 213+0+0 (unknown 3356586235 0 0) 0x7fe53c053f60 con 0x7fe544105f50 2024-02-14T17:59:06.839 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:06.838+0000 7fe54b8f6640 1 -- 172.21.15.178:0/2950442115 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"} v 0) v1 -- 0x7fe528001090 con 0x7fe544105f50 2024-02-14T17:59:07.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:06 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]: dispatch 2024-02-14T17:59:07.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:06 smithi178 ceph-mon[39811]: pgmap v1183: 176 pgs: 176 active+clean; 582 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:06 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]: dispatch 2024-02-14T17:59:07.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:06 smithi178 ceph-mon[30321]: pgmap v1183: 176 pgs: 176 active+clean; 582 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:07.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:06 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]: dispatch 2024-02-14T17:59:07.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:06 smithi184 ceph-mon[33536]: pgmap v1183: 176 pgs: 176 active+clean; 582 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:07.734 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:07.733+0000 7fe53a7fc640 1 -- 172.21.15.178:0/2950442115 <== mon.0 v1:172.21.15.178:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]=0 enabled application 'rados' on pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' v762) v1 ==== 213+0+0 (unknown 2255711185 0 0) 0x7fe53c055780 con 0x7fe544105f50 2024-02-14T17:59:07.735 INFO:tasks.workunit.client.0.smithi178.stderr:enabled application 'rados' on pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' 2024-02-14T17:59:07.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:07.737+0000 7fe54b8f6640 1 -- 172.21.15.178:0/2950442115 >> v1:172.21.15.178:6800/4205070280 conn(0x7fe53006e940 legacy=0x7fe530070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:07.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:07.737+0000 7fe54b8f6640 1 -- 172.21.15.178:0/2950442115 >> v1:172.21.15.178:6789/0 conn(0x7fe544105f50 legacy=0x7fe5441aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:07.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:07.737+0000 7fe54b8f6640 1 -- 172.21.15.178:0/2950442115 shutdown_connections 2024-02-14T17:59:07.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:07.737+0000 7fe54b8f6640 1 -- 172.21.15.178:0/2950442115 >> 172.21.15.178:0/2950442115 conn(0x7fe544075a30 msgr2=0x7fe5440ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:59:07.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:07.737+0000 7fe54b8f6640 1 -- 172.21.15.178:0/2950442115 shutdown_connections 2024-02-14T17:59:07.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:07.737+0000 7fe54b8f6640 1 -- 172.21.15.178:0/2950442115 wait complete. 2024-02-14T17:59:07.753 INFO:tasks.workunit.client.0.smithi178.stderr:++ seq 1 10 2024-02-14T17:59:07.835 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T17:59:07.835 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put obj1 /etc/passwd 2024-02-14T17:59:07.875 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T17:59:07.875 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put obj2 /etc/passwd 2024-02-14T17:59:07.910 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T17:59:07.911 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put obj3 /etc/passwd 2024-02-14T17:59:07.944 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T17:59:07.944 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put obj4 /etc/passwd 2024-02-14T17:59:07.978 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T17:59:07.978 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put obj5 /etc/passwd 2024-02-14T17:59:08.011 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T17:59:08.011 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put obj6 /etc/passwd 2024-02-14T17:59:08.044 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T17:59:08.044 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put obj7 /etc/passwd 2024-02-14T17:59:08.076 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T17:59:08.076 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put obj8 /etc/passwd 2024-02-14T17:59:08.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]': finished 2024-02-14T17:59:08.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:07 smithi178 ceph-mon[39811]: osdmap e761: 8 total, 8 up, 8 in 2024-02-14T17:59:08.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:07 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]: dispatch 2024-02-14T17:59:08.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]': finished 2024-02-14T17:59:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:07 smithi178 ceph-mon[30321]: osdmap e761: 8 total, 8 up, 8 in 2024-02-14T17:59:08.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:07 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]: dispatch 2024-02-14T17:59:08.109 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T17:59:08.109 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put obj9 /etc/passwd 2024-02-14T17:59:08.142 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T17:59:08.142 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put obj10 /etc/passwd 2024-02-14T17:59:08.174 INFO:tasks.workunit.client.0.smithi178.stderr:+ sleep 30 2024-02-14T17:59:08.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]': finished 2024-02-14T17:59:08.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:07 smithi184 ceph-mon[33536]: osdmap e761: 8 total, 8 up, 8 in 2024-02-14T17:59:08.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:07 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]: dispatch 2024-02-14T17:59:09.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:08 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]': finished 2024-02-14T17:59:09.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:08 smithi178 ceph-mon[39811]: osdmap e762: 8 total, 8 up, 8 in 2024-02-14T17:59:09.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:08 smithi178 ceph-mon[39811]: pgmap v1186: 176 pgs: 176 active+clean; 582 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:09.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:08 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]': finished 2024-02-14T17:59:09.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:08 smithi178 ceph-mon[30321]: osdmap e762: 8 total, 8 up, 8 in 2024-02-14T17:59:09.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:08 smithi178 ceph-mon[30321]: pgmap v1186: 176 pgs: 176 active+clean; 582 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:08 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2950442115' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "app": "rados"}]': finished 2024-02-14T17:59:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:08 smithi184 ceph-mon[33536]: osdmap e762: 8 total, 8 up, 8 in 2024-02-14T17:59:09.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:08 smithi184 ceph-mon[33536]: pgmap v1186: 176 pgs: 176 active+clean; 582 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:10.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:09 smithi178 ceph-mon[39811]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:10.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:09 smithi178 ceph-mon[30321]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:10.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:09 smithi184 ceph-mon[33536]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:11.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:10 smithi178 ceph-mon[30321]: pgmap v1187: 176 pgs: 176 active+clean; 582 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-02-14T17:59:11.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:10 smithi178 ceph-mon[39811]: pgmap v1187: 176 pgs: 176 active+clean; 582 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-02-14T17:59:11.120 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:10 smithi184 ceph-mon[33536]: pgmap v1187: 176 pgs: 176 active+clean; 582 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-02-14T17:59:11.968 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:59:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:59:12.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:12 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:12.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:12 smithi178 ceph-mon[39811]: pgmap v1188: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 765 B/s rd, 18 KiB/s wr, 2 op/s 2024-02-14T17:59:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:12 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:12 smithi178 ceph-mon[30321]: pgmap v1188: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 765 B/s rd, 18 KiB/s wr, 2 op/s 2024-02-14T17:59:12.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:12 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:12 smithi184 ceph-mon[33536]: pgmap v1188: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 765 B/s rd, 18 KiB/s wr, 2 op/s 2024-02-14T17:59:14.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:13 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:13 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:14.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:13 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:15.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:14 smithi178 ceph-mon[39811]: pgmap v1189: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 15 KiB/s wr, 1 op/s 2024-02-14T17:59:15.157 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:14 smithi178 ceph-mon[30321]: pgmap v1189: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 15 KiB/s wr, 1 op/s 2024-02-14T17:59:15.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:14 smithi184 ceph-mon[33536]: pgmap v1189: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 15 KiB/s wr, 1 op/s 2024-02-14T17:59:15.984 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:59:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:59:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:16.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:16 smithi178 ceph-mon[30321]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' is full (reached quota's max_objects: 10) 2024-02-14T17:59:16.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:16 smithi178 ceph-mon[30321]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:59:16.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:16 smithi178 ceph-mon[30321]: osdmap e763: 8 total, 8 up, 8 in 2024-02-14T17:59:16.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:16 smithi178 ceph-mon[39811]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' is full (reached quota's max_objects: 10) 2024-02-14T17:59:16.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:16 smithi178 ceph-mon[39811]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:59:16.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:16 smithi178 ceph-mon[39811]: osdmap e763: 8 total, 8 up, 8 in 2024-02-14T17:59:16.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:16 smithi184 ceph-mon[33536]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' is full (reached quota's max_objects: 10) 2024-02-14T17:59:16.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:16 smithi184 ceph-mon[33536]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:59:16.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:16 smithi184 ceph-mon[33536]: osdmap e763: 8 total, 8 up, 8 in 2024-02-14T17:59:17.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:17 smithi184 ceph-mon[33536]: pgmap v1191: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-14T17:59:17.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:17 smithi178 ceph-mon[30321]: pgmap v1191: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-14T17:59:17.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:17 smithi178 ceph-mon[39811]: pgmap v1191: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-14T17:59:18.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:18 smithi178 ceph-mon[39811]: pgmap v1192: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T17:59:18.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:18 smithi178 ceph-mon[30321]: pgmap v1192: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T17:59:18.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:18 smithi184 ceph-mon[33536]: pgmap v1192: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T17:59:20.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:20 smithi178 ceph-mon[39811]: pgmap v1193: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T17:59:20.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:20 smithi178 ceph-mon[30321]: pgmap v1193: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T17:59:20.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:20 smithi184 ceph-mon[33536]: pgmap v1193: 176 pgs: 176 active+clean; 697 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T17:59:21.968 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:59:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:59:22.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:22 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:22.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:22 smithi178 ceph-mon[39811]: pgmap v1194: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:22 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:22.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:22 smithi178 ceph-mon[30321]: pgmap v1194: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:22 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:22.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:22 smithi184 ceph-mon[33536]: pgmap v1194: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:24 smithi178 ceph-mon[39811]: pgmap v1195: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:24.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:24 smithi178 ceph-mon[30321]: pgmap v1195: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:24 smithi184 ceph-mon[33536]: pgmap v1195: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:26.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:59:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:59:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:26.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:26 smithi178 ceph-mon[39811]: pgmap v1196: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 991 B/s rd, 0 op/s 2024-02-14T17:59:26.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:26 smithi178 ceph-mon[30321]: pgmap v1196: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 991 B/s rd, 0 op/s 2024-02-14T17:59:26.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:26 smithi184 ceph-mon[33536]: pgmap v1196: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 991 B/s rd, 0 op/s 2024-02-14T17:59:28.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:28 smithi178 ceph-mon[39811]: pgmap v1197: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:59:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:28 smithi178 ceph-mon[30321]: pgmap v1197: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:59:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:28 smithi184 ceph-mon[33536]: pgmap v1197: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:59:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:29 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:59:30.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:29 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:30.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:29 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:59:30.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:29 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:30.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:29 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:59:30.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:29 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:31.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:30 smithi178 ceph-mon[39811]: pgmap v1198: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:31.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:30 smithi178 ceph-mon[30321]: pgmap v1198: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:31.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:30 smithi184 ceph-mon[33536]: pgmap v1198: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:31.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:59:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:59:32.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:32 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:32 smithi178 ceph-mon[39811]: pgmap v1199: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:59:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:32 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:32 smithi178 ceph-mon[30321]: pgmap v1199: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:59:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:32 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:32 smithi184 ceph-mon[33536]: pgmap v1199: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:59:34.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:34 smithi178 ceph-mon[39811]: pgmap v1200: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:59:34.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:34 smithi178 ceph-mon[30321]: pgmap v1200: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:59:34.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:34 smithi184 ceph-mon[33536]: pgmap v1200: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:59:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:59:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:59:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:36.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:36 smithi178 ceph-mon[39811]: pgmap v1201: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:36.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:36 smithi178 ceph-mon[30321]: pgmap v1201: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:36.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:36 smithi184 ceph-mon[33536]: pgmap v1201: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:38.175 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=151646 2024-02-14T17:59:38.176 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool set-quota c8dfcd28-16bd-46b5-a2ee-f0743e289c3b max_objects 100 2024-02-14T17:59:38.176 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put onemore /etc/passwd 2024-02-14T17:59:38.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.268+0000 7f6403c15640 1 -- 172.21.15.178:0/3816821275 >> v1:172.21.15.178:6790/0 conn(0x7f63fc10da40 legacy=0x7f63fc10ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:38.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.268+0000 7f6403c15640 1 -- 172.21.15.178:0/3816821275 shutdown_connections 2024-02-14T17:59:38.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.268+0000 7f6403c15640 1 -- 172.21.15.178:0/3816821275 >> 172.21.15.178:0/3816821275 conn(0x7f63fc075a30 msgr2=0x7f63fc075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:59:38.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.268+0000 7f6403c15640 1 -- 172.21.15.178:0/3816821275 shutdown_connections 2024-02-14T17:59:38.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.268+0000 7f6403c15640 1 -- 172.21.15.178:0/3816821275 wait complete. 2024-02-14T17:59:38.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f6403c15640 1 Processor -- start 2024-02-14T17:59:38.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f6403c15640 1 -- start start 2024-02-14T17:59:38.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f6403c15640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f63fc1a5e80 con 0x7f63fc0684d0 2024-02-14T17:59:38.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f6403c15640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f63fc1a7080 con 0x7f63fc109f00 2024-02-14T17:59:38.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f6403c15640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f63fc1a8280 con 0x7f63fc10da40 2024-02-14T17:59:38.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f64019b1640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f63fc0684d0 0x7f63fc106400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:52010/0 (socket says 172.21.15.178:52010) 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f64019b1640 1 -- 172.21.15.178:0/2651736932 learned_addr learned my addr 172.21.15.178:0/2651736932 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476940819 0 0) 0x7f63fc1a5e80 con 0x7f63fc0684d0 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63e8003520 con 0x7f63fc0684d0 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872489511 0 0) 0x7f63fc1a8280 con 0x7f63fc10da40 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63fc1a5e80 con 0x7f63fc10da40 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157977130 0 0) 0x7f63fc1a7080 con 0x7f63fc109f00 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.269+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63fc1a8280 con 0x7f63fc109f00 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3135015151 0 0) 0x7f63e8003520 con 0x7f63fc0684d0 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63fc1a7080 con 0x7f63fc0684d0 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f63f40012d0 con 0x7f63fc0684d0 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2211417030 0 0) 0x7f63fc1a5e80 con 0x7f63fc10da40 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63e8003520 con 0x7f63fc10da40 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f63f8001ab0 con 0x7f63fc10da40 2024-02-14T17:59:38.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1892844510 0 0) 0x7f63fc1a8280 con 0x7f63fc109f00 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63fc1a5e80 con 0x7f63fc109f00 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f63ec001870 con 0x7f63fc109f00 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3455450210 0 0) 0x7f63fc1a7080 con 0x7f63fc0684d0 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 >> v1:172.21.15.178:6790/0 conn(0x7f63fc10da40 legacy=0x7f63fc1a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 >> v1:172.21.15.184:6789/0 conn(0x7f63fc109f00 legacy=0x7f63fc1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc1a9480 con 0x7f63fc0684d0 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f6403c15640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63fc1a8470 con 0x7f63fc0684d0 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f6403c15640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f63fc1a8900 con 0x7f63fc0684d0 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f63f4003a80 con 0x7f63fc0684d0 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.270+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f63f4005af0 con 0x7f63fc0684d0 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.271+0000 7f6403c15640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e0005080 con 0x7f63fc0684d0 2024-02-14T17:59:38.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.271+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f63f40035f0 con 0x7f63fc0684d0 2024-02-14T17:59:38.276 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.272+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(763..763 src has 255..763) v4 ==== 7830+0+0 (unknown 1368894485 0 0) 0x7f63f4088e20 con 0x7f63fc0684d0 2024-02-14T17:59:38.276 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.272+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=764}) v3 -- 0x7f63fc1a7080 con 0x7f63fc0684d0 2024-02-14T17:59:38.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.276+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f63f4055b00 con 0x7f63fc0684d0 2024-02-14T17:59:38.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.454+0000 7f6403c15640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f63e0005330 con 0x7f63fc0684d0 2024-02-14T17:59:38.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.506+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v764) v1 ==== 225+0+0 (unknown 1174708464 0 0) 0x7f63f4053e70 con 0x7f63fc0684d0 2024-02-14T17:59:38.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.507+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 11 ==== osd_map(764..764 src has 255..764) v4 ==== 628+0+0 (unknown 696868582 0 0) 0x7f63f4056430 con 0x7f63fc0684d0 2024-02-14T17:59:38.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.507+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=765}) v3 -- 0x7f63fc1a8280 con 0x7f63fc0684d0 2024-02-14T17:59:38.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:38.614+0000 7f6403c15640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f63e0001090 con 0x7f63fc0684d0 2024-02-14T17:59:38.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:38 smithi178 ceph-mon[39811]: pgmap v1202: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:59:38.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:38 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T17:59:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:38 smithi178 ceph-mon[30321]: pgmap v1202: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:59:38.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:38 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T17:59:38.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:38 smithi184 ceph-mon[33536]: pgmap v1202: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:59:38.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:38 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T17:59:39.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:39.508+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v765) v1 ==== 225+0+0 (unknown 1860672648 0 0) 0x7f63f4055690 con 0x7f63fc0684d0 2024-02-14T17:59:39.510 INFO:tasks.workunit.client.0.smithi178.stderr:set-quota max_objects = 100 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b 2024-02-14T17:59:39.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:39.508+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 <== mon.0 v1:172.21.15.178:6789/0 13 ==== osd_map(765..765 src has 255..765) v4 ==== 628+0+0 (unknown 2034688914 0 0) 0x7f63f4059e80 con 0x7f63fc0684d0 2024-02-14T17:59:39.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:39.508+0000 7f63f2ffd640 1 -- 172.21.15.178:0/2651736932 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=766}) v3 -- 0x7f63e807bbe0 con 0x7f63fc0684d0 2024-02-14T17:59:39.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:39.515+0000 7f6403c15640 1 -- 172.21.15.178:0/2651736932 >> v1:172.21.15.178:6800/4205070280 conn(0x7f63e806ea90 legacy=0x7f63e8070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:39.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:39.515+0000 7f6403c15640 1 -- 172.21.15.178:0/2651736932 >> v1:172.21.15.178:6789/0 conn(0x7f63fc0684d0 legacy=0x7f63fc106400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:39.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:39.515+0000 7f6403c15640 1 -- 172.21.15.178:0/2651736932 shutdown_connections 2024-02-14T17:59:39.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:39.515+0000 7f6403c15640 1 -- 172.21.15.178:0/2651736932 >> 172.21.15.178:0/2651736932 conn(0x7f63fc075a30 msgr2=0x7f63fc10fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:59:39.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:39.515+0000 7f6403c15640 1 -- 172.21.15.178:0/2651736932 shutdown_connections 2024-02-14T17:59:39.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:39.515+0000 7f6403c15640 1 -- 172.21.15.178:0/2651736932 wait complete. 2024-02-14T17:59:39.528 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 151646 2024-02-14T17:59:39.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]': finished 2024-02-14T17:59:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:39 smithi178 ceph-mon[39811]: osdmap e764: 8 total, 8 up, 8 in 2024-02-14T17:59:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:39 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T17:59:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]': finished 2024-02-14T17:59:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:39 smithi178 ceph-mon[30321]: osdmap e764: 8 total, 8 up, 8 in 2024-02-14T17:59:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:39 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T17:59:39.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]': finished 2024-02-14T17:59:39.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:39 smithi184 ceph-mon[33536]: osdmap e764: 8 total, 8 up, 8 in 2024-02-14T17:59:39.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:39 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T17:59:40.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:40 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]': finished 2024-02-14T17:59:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:40 smithi178 ceph-mon[39811]: osdmap e765: 8 total, 8 up, 8 in 2024-02-14T17:59:40.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:40 smithi178 ceph-mon[39811]: pgmap v1205: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:40 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]': finished 2024-02-14T17:59:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:40 smithi178 ceph-mon[30321]: osdmap e765: 8 total, 8 up, 8 in 2024-02-14T17:59:40.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:40 smithi178 ceph-mon[30321]: pgmap v1205: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:40.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:40 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/2651736932' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "100"}]': finished 2024-02-14T17:59:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:40 smithi184 ceph-mon[33536]: osdmap e765: 8 total, 8 up, 8 in 2024-02-14T17:59:40.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:40 smithi184 ceph-mon[33536]: pgmap v1205: 176 pgs: 176 active+clean; 697 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:41.107 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 0 -ne 0 ']' 2024-02-14T17:59:41.107 INFO:tasks.workunit.client.0.smithi178.stderr:+ true 2024-02-14T17:59:41.107 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put twomore /etc/passwd 2024-02-14T17:59:41.145 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool set-quota c8dfcd28-16bd-46b5-a2ee-f0743e289c3b max_bytes 100 2024-02-14T17:59:41.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.226+0000 7f899ef58640 1 -- 172.21.15.178:0/3236507086 >> v1:172.21.15.178:6790/0 conn(0x7f8998100160 legacy=0x7f8998100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:41.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.226+0000 7f899ef58640 1 -- 172.21.15.178:0/3236507086 shutdown_connections 2024-02-14T17:59:41.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.226+0000 7f899ef58640 1 -- 172.21.15.178:0/3236507086 >> 172.21.15.178:0/3236507086 conn(0x7f89980fb9e0 msgr2=0x7f89980fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:59:41.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.226+0000 7f899ef58640 1 -- 172.21.15.178:0/3236507086 shutdown_connections 2024-02-14T17:59:41.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.226+0000 7f899ef58640 1 -- 172.21.15.178:0/3236507086 wait complete. 2024-02-14T17:59:41.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.226+0000 7f899ef58640 1 Processor -- start 2024-02-14T17:59:41.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.226+0000 7f899ef58640 1 -- start start 2024-02-14T17:59:41.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.226+0000 7f899ef58640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f89981a4da0 con 0x7f8998100160 2024-02-14T17:59:41.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.226+0000 7f899ef58640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f89981a5fa0 con 0x7f899810c950 2024-02-14T17:59:41.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.226+0000 7f899ef58640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f89981a71a0 con 0x7f8998108de0 2024-02-14T17:59:41.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8997fff640 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f8998108de0 0x7f899819fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:50704/0 (socket says 172.21.15.178:50704) 2024-02-14T17:59:41.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f899ccf4640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f8998100160 0x7f89981053c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:52072/0 (socket says 172.21.15.178:52072) 2024-02-14T17:59:41.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8997fff640 1 -- 172.21.15.178:0/1033190528 learned_addr learned my addr 172.21.15.178:0/1033190528 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T17:59:41.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136859964 0 0) 0x7f89981a71a0 con 0x7f8998108de0 2024-02-14T17:59:41.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8988003520 con 0x7f8998108de0 2024-02-14T17:59:41.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336866856 0 0) 0x7f89981a4da0 con 0x7f8998100160 2024-02-14T17:59:41.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89981a71a0 con 0x7f8998100160 2024-02-14T17:59:41.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450989416 0 0) 0x7f89981a5fa0 con 0x7f899810c950 2024-02-14T17:59:41.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89981a4da0 con 0x7f899810c950 2024-02-14T17:59:41.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1817536797 0 0) 0x7f8988003520 con 0x7f8998108de0 2024-02-14T17:59:41.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f89981a5fa0 con 0x7f8998108de0 2024-02-14T17:59:41.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f89840018b0 con 0x7f8998108de0 2024-02-14T17:59:41.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3003119690 0 0) 0x7f89981a71a0 con 0x7f8998100160 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8988003520 con 0x7f8998100160 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f898c0043a0 con 0x7f8998100160 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3336051844 0 0) 0x7f89981a4da0 con 0x7f899810c950 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f89981a71a0 con 0x7f899810c950 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f8990001b50 con 0x7f899810c950 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3124529483 0 0) 0x7f89981a5fa0 con 0x7f8998108de0 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 >> v1:172.21.15.184:6789/0 conn(0x7f899810c950 legacy=0x7f89981a34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 >> v1:172.21.15.178:6789/0 conn(0x7f8998100160 legacy=0x7f89981053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89981a83a0 con 0x7f8998108de0 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f899ef58640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89981a7390 con 0x7f8998108de0 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f8984003aa0 con 0x7f8998108de0 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f8984005ba0 con 0x7f8998108de0 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.227+0000 7f899ef58640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f89981a7820 con 0x7f8998108de0 2024-02-14T17:59:41.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.228+0000 7f899ef58640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8998101630 con 0x7f8998108de0 2024-02-14T17:59:41.233 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.229+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f8984003610 con 0x7f8998108de0 2024-02-14T17:59:41.233 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.229+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(766..766 src has 255..766) v4 ==== 7830+0+0 (unknown 1411977705 0 0) 0x7f8984088df0 con 0x7f8998108de0 2024-02-14T17:59:41.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.234+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f8984055ad0 con 0x7f8998108de0 2024-02-14T17:59:41.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:41.409+0000 7f899ef58640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f8998068260 con 0x7f8998108de0 2024-02-14T17:59:41.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:41 smithi178 ceph-mon[39811]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' no longer out of quota; removing NO_QUOTA flag 2024-02-14T17:59:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:41 smithi178 ceph-mon[39811]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T17:59:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:41 smithi178 ceph-mon[39811]: osdmap e766: 8 total, 8 up, 8 in 2024-02-14T17:59:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:41 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1033190528' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:41.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:41 smithi178 ceph-mon[39811]: from='client.57101 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:41 smithi178 ceph-mon[30321]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' no longer out of quota; removing NO_QUOTA flag 2024-02-14T17:59:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:41 smithi178 ceph-mon[30321]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T17:59:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:41 smithi178 ceph-mon[30321]: osdmap e766: 8 total, 8 up, 8 in 2024-02-14T17:59:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:41 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1033190528' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:41.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:41 smithi178 ceph-mon[30321]: from='client.57101 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:41.870 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:59:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:59:41.871 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:41 smithi184 ceph-mon[33536]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' no longer out of quota; removing NO_QUOTA flag 2024-02-14T17:59:41.871 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:41 smithi184 ceph-mon[33536]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T17:59:41.871 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:41 smithi184 ceph-mon[33536]: osdmap e766: 8 total, 8 up, 8 in 2024-02-14T17:59:41.871 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:41 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1033190528' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:41.871 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:41 smithi184 ceph-mon[33536]: from='client.57101 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:42.101 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:42.099+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.2 v1:172.21.15.178:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v767) v1 ==== 221+0+0 (unknown 2685418989 0 0) 0x7f8984053e40 con 0x7f8998108de0 2024-02-14T17:59:42.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:42.207+0000 7f899ef58640 1 -- 172.21.15.178:0/1033190528 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f8998103990 con 0x7f8998108de0 2024-02-14T17:59:43.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:43.107+0000 7f8995ffb640 1 -- 172.21.15.178:0/1033190528 <== mon.2 v1:172.21.15.178:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v768) v1 ==== 221+0+0 (unknown 369349902 0 0) 0x7f8984055660 con 0x7f8998108de0 2024-02-14T17:59:43.108 INFO:tasks.workunit.client.0.smithi178.stderr:set-quota max_bytes = 100 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b 2024-02-14T17:59:43.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:43.109+0000 7f899ef58640 1 -- 172.21.15.178:0/1033190528 >> v1:172.21.15.178:6800/4205070280 conn(0x7f898806ea50 legacy=0x7f8988070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:43.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:43.109+0000 7f899ef58640 1 -- 172.21.15.178:0/1033190528 >> v1:172.21.15.178:6790/0 conn(0x7f8998108de0 legacy=0x7f899819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:59:43.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:43.109+0000 7f899ef58640 1 -- 172.21.15.178:0/1033190528 shutdown_connections 2024-02-14T17:59:43.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:43.109+0000 7f899ef58640 1 -- 172.21.15.178:0/1033190528 >> 172.21.15.178:0/1033190528 conn(0x7f89980fb9e0 msgr2=0x7f899810f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:59:43.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:43.109+0000 7f899ef58640 1 -- 172.21.15.178:0/1033190528 shutdown_connections 2024-02-14T17:59:43.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T17:59:43.109+0000 7f899ef58640 1 -- 172.21.15.178:0/1033190528 wait complete. 2024-02-14T17:59:43.122 INFO:tasks.workunit.client.0.smithi178.stderr:+ sleep 30 2024-02-14T17:59:43.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:43 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:43.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:43 smithi184 ceph-mon[33536]: from='client.57101 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T17:59:43.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:43 smithi184 ceph-mon[33536]: osdmap e767: 8 total, 8 up, 8 in 2024-02-14T17:59:43.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:43 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1033190528' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:43.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:43 smithi184 ceph-mon[33536]: from='client.57101 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:43.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:43 smithi184 ceph-mon[33536]: pgmap v1208: 176 pgs: 176 active+clean; 697 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:43.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:43.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[39811]: from='client.57101 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T17:59:43.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[39811]: osdmap e767: 8 total, 8 up, 8 in 2024-02-14T17:59:43.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1033190528' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:43.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[39811]: from='client.57101 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:43.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[39811]: pgmap v1208: 176 pgs: 176 active+clean; 697 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[30321]: from='client.57101 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T17:59:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[30321]: osdmap e767: 8 total, 8 up, 8 in 2024-02-14T17:59:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1033190528' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[30321]: from='client.57101 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T17:59:43.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:43 smithi178 ceph-mon[30321]: pgmap v1208: 176 pgs: 176 active+clean; 697 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:44.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:44 smithi184 ceph-mon[33536]: from='client.57101 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T17:59:44.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:44 smithi184 ceph-mon[33536]: osdmap e768: 8 total, 8 up, 8 in 2024-02-14T17:59:44.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:44 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:59:44.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:44.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:44 smithi178 ceph-mon[39811]: from='client.57101 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T17:59:44.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:44 smithi178 ceph-mon[39811]: osdmap e768: 8 total, 8 up, 8 in 2024-02-14T17:59:44.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:44 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:59:44.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:44.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:44 smithi178 ceph-mon[30321]: from='client.57101 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T17:59:44.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:44 smithi178 ceph-mon[30321]: osdmap e768: 8 total, 8 up, 8 in 2024-02-14T17:59:44.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:44 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:59:44.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:45.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:45 smithi184 ceph-mon[33536]: pgmap v1210: 176 pgs: 176 active+clean; 697 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-14T17:59:45.483 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:45 smithi178 ceph-mon[39811]: pgmap v1210: 176 pgs: 176 active+clean; 697 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-14T17:59:45.484 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:45 smithi178 ceph-mon[30321]: pgmap v1210: 176 pgs: 176 active+clean; 697 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-14T17:59:45.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:59:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:59:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:46.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:46 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:59:46.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:46 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:59:46.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:46 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:59:46.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:46 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:59:46.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:46 smithi184 ceph-mon[33536]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' is full (reached quota's max_bytes: 100 B) 2024-02-14T17:59:46.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:46 smithi184 ceph-mon[33536]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:59:46.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:46 smithi184 ceph-mon[33536]: osdmap e769: 8 total, 8 up, 8 in 2024-02-14T17:59:46.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:59:46.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:59:46.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:59:46.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:59:46.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[30321]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' is full (reached quota's max_bytes: 100 B) 2024-02-14T17:59:46.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[30321]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:59:46.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[30321]: osdmap e769: 8 total, 8 up, 8 in 2024-02-14T17:59:46.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:59:46.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:59:46.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:59:46.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T17:59:46.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[39811]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' is full (reached quota's max_bytes: 100 B) 2024-02-14T17:59:46.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[39811]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:59:46.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:46 smithi178 ceph-mon[39811]: osdmap e769: 8 total, 8 up, 8 in 2024-02-14T17:59:47.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:47 smithi184 ceph-mon[33536]: pgmap v1212: 176 pgs: 176 active+clean; 720 KiB data, 837 MiB used, 714 GiB / 715 GiB avail; 961 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-02-14T17:59:47.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:47 smithi178 ceph-mon[30321]: pgmap v1212: 176 pgs: 176 active+clean; 720 KiB data, 837 MiB used, 714 GiB / 715 GiB avail; 961 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-02-14T17:59:47.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:47 smithi178 ceph-mon[39811]: pgmap v1212: 176 pgs: 176 active+clean; 720 KiB data, 837 MiB used, 714 GiB / 715 GiB avail; 961 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-02-14T17:59:48.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:48 smithi178 ceph-mon[39811]: pgmap v1213: 176 pgs: 176 active+clean; 720 KiB data, 837 MiB used, 714 GiB / 715 GiB avail; 809 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-02-14T17:59:48.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:48 smithi178 ceph-mon[30321]: pgmap v1213: 176 pgs: 176 active+clean; 720 KiB data, 837 MiB used, 714 GiB / 715 GiB avail; 809 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-02-14T17:59:48.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:48 smithi184 ceph-mon[33536]: pgmap v1213: 176 pgs: 176 active+clean; 720 KiB data, 837 MiB used, 714 GiB / 715 GiB avail; 809 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-02-14T17:59:50.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:50 smithi178 ceph-mon[39811]: pgmap v1214: 176 pgs: 176 active+clean; 720 KiB data, 837 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-14T17:59:50.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:50 smithi178 ceph-mon[30321]: pgmap v1214: 176 pgs: 176 active+clean; 720 KiB data, 837 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-14T17:59:50.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:50 smithi184 ceph-mon[33536]: pgmap v1214: 176 pgs: 176 active+clean; 720 KiB data, 837 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-14T17:59:52.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 17:59:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T17:59:52.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:52 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:52.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:52 smithi178 ceph-mon[39811]: pgmap v1215: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-02-14T17:59:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:52 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:52 smithi178 ceph-mon[30321]: pgmap v1215: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-02-14T17:59:52.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:52 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:52 smithi184 ceph-mon[33536]: pgmap v1215: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-02-14T17:59:54.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:54 smithi178 ceph-mon[39811]: pgmap v1216: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-14T17:59:54.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:54 smithi178 ceph-mon[30321]: pgmap v1216: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-14T17:59:54.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:54 smithi184 ceph-mon[33536]: pgmap v1216: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-14T17:59:56.083 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 17:59:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:17:59:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:56.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:56 smithi178 ceph-mon[30321]: pgmap v1217: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 991 B/s rd, 0 op/s 2024-02-14T17:59:56.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:56 smithi178 ceph-mon[39811]: pgmap v1217: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 991 B/s rd, 0 op/s 2024-02-14T17:59:56.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:56 smithi184 ceph-mon[33536]: pgmap v1217: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 991 B/s rd, 0 op/s 2024-02-14T17:59:58.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:58 smithi178 ceph-mon[30321]: pgmap v1218: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:59:58.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:58 smithi178 ceph-mon[39811]: pgmap v1218: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:59:58.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:58 smithi184 ceph-mon[33536]: pgmap v1218: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:00.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:59 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:00:00.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 17:59:59 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:59 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:00:00.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 17:59:59 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:00.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:59 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:00:00.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 17:59:59 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:01.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[39811]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-02-14T18:00:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[39811]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-14T18:00:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[39811]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:00:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[39811]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:00:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[39811]: [WRN] POOL_FULL: 1 pool(s) full 2024-02-14T18:00:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[39811]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' is full (running out of quota) 2024-02-14T18:00:01.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[39811]: pgmap v1219: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[30321]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-02-14T18:00:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[30321]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-14T18:00:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[30321]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:00:01.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[30321]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:00:01.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[30321]: [WRN] POOL_FULL: 1 pool(s) full 2024-02-14T18:00:01.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[30321]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' is full (running out of quota) 2024-02-14T18:00:01.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:00 smithi178 ceph-mon[30321]: pgmap v1219: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:00 smithi184 ceph-mon[33536]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-02-14T18:00:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:00 smithi184 ceph-mon[33536]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-14T18:00:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:00 smithi184 ceph-mon[33536]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:00:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:00 smithi184 ceph-mon[33536]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:00:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:00 smithi184 ceph-mon[33536]: [WRN] POOL_FULL: 1 pool(s) full 2024-02-14T18:00:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:00 smithi184 ceph-mon[33536]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' is full (running out of quota) 2024-02-14T18:00:01.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:00 smithi184 ceph-mon[33536]: pgmap v1219: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:01.983 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:00:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:00:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:02 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:02.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:02 smithi178 ceph-mon[39811]: pgmap v1220: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:02 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:02 smithi178 ceph-mon[30321]: pgmap v1220: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:02.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:02 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:02.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:02 smithi184 ceph-mon[33536]: pgmap v1220: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:04.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:04 smithi178 ceph-mon[30321]: pgmap v1221: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:04.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:04 smithi178 ceph-mon[39811]: pgmap v1221: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:04.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:04 smithi184 ceph-mon[33536]: pgmap v1221: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:06.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:00:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:00:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:06.834 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:06 smithi178 ceph-mon[39811]: pgmap v1222: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:06.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:06 smithi178 ceph-mon[30321]: pgmap v1222: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:06.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:06 smithi184 ceph-mon[33536]: pgmap v1222: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:08.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:08 smithi178 ceph-mon[39811]: pgmap v1223: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:08.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:08 smithi178 ceph-mon[30321]: pgmap v1223: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:08.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:08 smithi184 ceph-mon[33536]: pgmap v1223: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:10.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:10 smithi178 ceph-mon[39811]: pgmap v1224: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:10.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:10 smithi178 ceph-mon[30321]: pgmap v1224: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:10.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:10 smithi184 ceph-mon[33536]: pgmap v1224: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:12.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:00:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:00:12.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:12 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:12.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:12 smithi178 ceph-mon[30321]: pgmap v1225: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:12 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:12.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:12 smithi178 ceph-mon[39811]: pgmap v1225: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:12.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:12 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:12.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:12 smithi184 ceph-mon[33536]: pgmap v1225: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:+ pid=151866 2024-02-14T18:00:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool set-quota c8dfcd28-16bd-46b5-a2ee-f0743e289c3b max_bytes 0 2024-02-14T18:00:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put two /etc/passwd 2024-02-14T18:00:13.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.216+0000 7f617feed640 1 -- 172.21.15.178:0/1772361018 >> v1:172.21.15.178:6790/0 conn(0x7f61780fff90 legacy=0x7f6178100370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:13.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.216+0000 7f617feed640 1 -- 172.21.15.178:0/1772361018 shutdown_connections 2024-02-14T18:00:13.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.216+0000 7f617feed640 1 -- 172.21.15.178:0/1772361018 >> 172.21.15.178:0/1772361018 conn(0x7f61780fb9a0 msgr2=0x7f61780fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:00:13.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.216+0000 7f617feed640 1 -- 172.21.15.178:0/1772361018 shutdown_connections 2024-02-14T18:00:13.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.216+0000 7f617feed640 1 -- 172.21.15.178:0/1772361018 wait complete. 2024-02-14T18:00:13.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.216+0000 7f617feed640 1 Processor -- start 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.216+0000 7f617feed640 1 -- start start 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f617feed640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f61781ac510 con 0x7f6178114290 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f617feed640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f61781ad6d0 con 0x7f61780fff90 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f617feed640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f61781ae8d0 con 0x7f6178105d80 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f617d488640 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f6178105d80 0x7f61781a73b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:42418/0 (socket says 172.21.15.178:42418) 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f617d488640 1 -- 172.21.15.178:0/3163957618 learned_addr learned my addr 172.21.15.178:0/3163957618 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757415655 0 0) 0x7f61781ae8d0 con 0x7f6178105d80 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6164003520 con 0x7f6178105d80 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927583406 0 0) 0x7f61781ac510 con 0x7f6178114290 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61781ae8d0 con 0x7f6178114290 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947585255 0 0) 0x7f61781ad6d0 con 0x7f61780fff90 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61781ac510 con 0x7f61780fff90 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4032776672 0 0) 0x7f6164003520 con 0x7f6178105d80 2024-02-14T18:00:13.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f61781ad6d0 con 0x7f6178105d80 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f6168001930 con 0x7f6178105d80 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4164598580 0 0) 0x7f61781ae8d0 con 0x7f6178114290 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6164003520 con 0x7f6178114290 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3369416235 0 0) 0x7f61781ac510 con 0x7f61780fff90 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f61781ae8d0 con 0x7f61780fff90 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f61700013a0 con 0x7f6178114290 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f6174001880 con 0x7f61780fff90 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3348638670 0 0) 0x7f61781ad6d0 con 0x7f6178105d80 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 >> v1:172.21.15.184:6789/0 conn(0x7f61780fff90 legacy=0x7f6178113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 >> v1:172.21.15.178:6789/0 conn(0x7f6178114290 legacy=0x7f61781aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61781afad0 con 0x7f6178105d80 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.217+0000 7f617feed640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61781ac700 con 0x7f6178105d80 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.218+0000 7f617feed640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f61781acb90 con 0x7f6178105d80 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.218+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f6168003aa0 con 0x7f6178105d80 2024-02-14T18:00:13.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.218+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f6168005c10 con 0x7f6178105d80 2024-02-14T18:00:13.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.219+0000 7f617feed640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615c005080 con 0x7f6178105d80 2024-02-14T18:00:13.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.219+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f616801c360 con 0x7f6178105d80 2024-02-14T18:00:13.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.219+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(769..769 src has 255..769) v4 ==== 7830+0+0 (unknown 1773674739 0 0) 0x7f6168088f30 con 0x7f6178105d80 2024-02-14T18:00:13.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.219+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=770}) v3 -- 0x7f61781ad6d0 con 0x7f6178105d80 2024-02-14T18:00:13.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.224+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f6168055980 con 0x7f6178105d80 2024-02-14T18:00:13.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.400+0000 7f617feed640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f615c005330 con 0x7f6178105d80 2024-02-14T18:00:13.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.493+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 10 ==== osd_map(770..770 src has 255..770) v4 ==== 628+0+0 (unknown 1525734788 0 0) 0x7f6168056540 con 0x7f6178105d80 2024-02-14T18:00:13.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.493+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=771}) v3 -- 0x7f61781ac510 con 0x7f6178105d80 2024-02-14T18:00:13.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.498+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v770) v1 ==== 217+0+0 (unknown 400172644 0 0) 0x7f6168053f80 con 0x7f6178105d80 2024-02-14T18:00:13.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:13.603+0000 7f617feed640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f615c000c40 con 0x7f6178105d80 2024-02-14T18:00:13.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:13 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3163957618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:13.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:13 smithi178 ceph-mon[39811]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:13 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3163957618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:13 smithi178 ceph-mon[30321]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:13 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3163957618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:13 smithi184 ceph-mon[33536]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:14.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.499+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 12 ==== osd_map(771..771 src has 255..771) v4 ==== 628+0+0 (unknown 1051390218 0 0) 0x7f6168056540 con 0x7f6178105d80 2024-02-14T18:00:14.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.499+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=772}) v3 -- 0x7f616407bd70 con 0x7f6178105d80 2024-02-14T18:00:14.505 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.503+0000 7f616effd640 1 -- 172.21.15.178:0/3163957618 <== mon.2 v1:172.21.15.178:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v771) v1 ==== 217+0+0 (unknown 3330354090 0 0) 0x7f61680557a0 con 0x7f6178105d80 2024-02-14T18:00:14.505 INFO:tasks.workunit.client.0.smithi178.stderr:set-quota max_bytes = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b 2024-02-14T18:00:14.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.507+0000 7f617feed640 1 -- 172.21.15.178:0/3163957618 >> v1:172.21.15.178:6800/4205070280 conn(0x7f616406ec60 legacy=0x7f6164071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:14.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.507+0000 7f617feed640 1 -- 172.21.15.178:0/3163957618 >> v1:172.21.15.178:6790/0 conn(0x7f6178105d80 legacy=0x7f61781a73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:14.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.507+0000 7f617feed640 1 -- 172.21.15.178:0/3163957618 shutdown_connections 2024-02-14T18:00:14.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.507+0000 7f617feed640 1 -- 172.21.15.178:0/3163957618 >> 172.21.15.178:0/3163957618 conn(0x7f61780fb9a0 msgr2=0x7f61780fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:00:14.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.507+0000 7f617feed640 1 -- 172.21.15.178:0/3163957618 shutdown_connections 2024-02-14T18:00:14.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.507+0000 7f617feed640 1 -- 172.21.15.178:0/3163957618 wait complete. 2024-02-14T18:00:14.519 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool set-quota c8dfcd28-16bd-46b5-a2ee-f0743e289c3b max_objects 0 2024-02-14T18:00:14.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.602+0000 7f0628530640 1 -- 172.21.15.178:0/1638489750 >> v1:172.21.15.178:6790/0 conn(0x7f062010c930 legacy=0x7f062010edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:14.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.602+0000 7f0628530640 1 -- 172.21.15.178:0/1638489750 shutdown_connections 2024-02-14T18:00:14.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.602+0000 7f0628530640 1 -- 172.21.15.178:0/1638489750 >> 172.21.15.178:0/1638489750 conn(0x7f0620075a30 msgr2=0x7f0620075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:00:14.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.602+0000 7f0628530640 1 -- 172.21.15.178:0/1638489750 shutdown_connections 2024-02-14T18:00:14.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.602+0000 7f0628530640 1 -- 172.21.15.178:0/1638489750 wait complete. 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.602+0000 7f0628530640 1 Processor -- start 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f0628530640 1 -- start start 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f0628530640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f06201a5f40 con 0x7f0620100280 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f0628530640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f06201a7140 con 0x7f062010c930 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f0628530640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f06201a8340 con 0x7f0620108dc0 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f0625acb640 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f0620108dc0 0x7f06201a0cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:49594/0 (socket says 172.21.15.178:49594) 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f0625acb640 1 -- 172.21.15.178:0/3530779301 learned_addr learned my addr 172.21.15.178:0/3530779301 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758036609 0 0) 0x7f06201a8340 con 0x7f0620108dc0 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f060c003520 con 0x7f0620108dc0 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672457260 0 0) 0x7f06201a5f40 con 0x7f0620100280 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06201a8340 con 0x7f0620100280 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22844715 0 0) 0x7f06201a7140 con 0x7f062010c930 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06201a5f40 con 0x7f062010c930 2024-02-14T18:00:14.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1229930820 0 0) 0x7f060c003520 con 0x7f0620108dc0 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f06201a7140 con 0x7f0620108dc0 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1350771276 0 0) 0x7f06201a8340 con 0x7f0620100280 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f060c003520 con 0x7f0620100280 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f06100018d0 con 0x7f0620108dc0 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f061c001bb0 con 0x7f0620100280 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3775150527 0 0) 0x7f06201a7140 con 0x7f0620108dc0 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 >> v1:172.21.15.184:6789/0 conn(0x7f062010c930 legacy=0x7f06201a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 >> v1:172.21.15.178:6789/0 conn(0x7f0620100280 legacy=0x7f0620068c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06201a9540 con 0x7f0620108dc0 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.603+0000 7f0628530640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06201a6130 con 0x7f0620108dc0 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.604+0000 7f0628530640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f06201a65c0 con 0x7f0620108dc0 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.604+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f0610003a80 con 0x7f0620108dc0 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.604+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f0610005b30 con 0x7f0620108dc0 2024-02-14T18:00:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.604+0000 7f0628530640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0604005080 con 0x7f0620108dc0 2024-02-14T18:00:14.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.605+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f06100035f0 con 0x7f0620108dc0 2024-02-14T18:00:14.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.605+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(771..771 src has 255..771) v4 ==== 7830+0+0 (unknown 2538619076 0 0) 0x7f0610088e30 con 0x7f0620108dc0 2024-02-14T18:00:14.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.605+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=772}) v3 -- 0x7f06201a7140 con 0x7f0620108dc0 2024-02-14T18:00:14.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.610+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f0610055bf0 con 0x7f0620108dc0 2024-02-14T18:00:14.786 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:14.784+0000 7f0628530640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f0604005330 con 0x7f0620108dc0 2024-02-14T18:00:14.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[39811]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:00:14.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[39811]: osdmap e770: 8 total, 8 up, 8 in 2024-02-14T18:00:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3163957618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[39811]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:14.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[39811]: pgmap v1227: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:00:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[30321]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:00:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[30321]: osdmap e770: 8 total, 8 up, 8 in 2024-02-14T18:00:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3163957618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[30321]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:14 smithi178 ceph-mon[30321]: pgmap v1227: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:00:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:14 smithi184 ceph-mon[33536]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:00:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:14 smithi184 ceph-mon[33536]: osdmap e770: 8 total, 8 up, 8 in 2024-02-14T18:00:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:14 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3163957618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:14 smithi184 ceph-mon[33536]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:00:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:14 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:14.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:14 smithi184 ceph-mon[33536]: pgmap v1227: 176 pgs: 176 active+clean; 720 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:00:15.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:15.501+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 10 ==== osd_map(772..772 src has 255..772) v4 ==== 628+0+0 (unknown 3023505774 0 0) 0x7f0610056520 con 0x7f0620108dc0 2024-02-14T18:00:15.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:15.501+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=773}) v3 -- 0x7f060c003520 con 0x7f0620108dc0 2024-02-14T18:00:15.505 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:15.504+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v772) v1 ==== 221+0+0 (unknown 2074279814 0 0) 0x7f0610053f60 con 0x7f0620108dc0 2024-02-14T18:00:15.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:15.617+0000 7f0628530640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f06040010c0 con 0x7f0620108dc0 2024-02-14T18:00:15.832 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:00:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:00:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:15.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:15 smithi178 ceph-mon[39811]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:00:15.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:15 smithi178 ceph-mon[39811]: osdmap e771: 8 total, 8 up, 8 in 2024-02-14T18:00:15.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:15 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3530779301' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:15.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:15 smithi178 ceph-mon[39811]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:15.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:15 smithi178 ceph-mon[30321]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:00:15.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:15 smithi178 ceph-mon[30321]: osdmap e771: 8 total, 8 up, 8 in 2024-02-14T18:00:15.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:15 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3530779301' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:15.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:15 smithi178 ceph-mon[30321]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:15 smithi184 ceph-mon[33536]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:00:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:15 smithi184 ceph-mon[33536]: osdmap e771: 8 total, 8 up, 8 in 2024-02-14T18:00:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:15 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3530779301' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:15.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:15 smithi184 ceph-mon[33536]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:16.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.100+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 12 ==== osd_map(773..773 src has 255..773) v4 ==== 628+0+0 (unknown 1500008887 0 0) 0x7f0610059f70 con 0x7f0620108dc0 2024-02-14T18:00:16.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.100+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=774}) v3 -- 0x7f06201a8340 con 0x7f0620108dc0 2024-02-14T18:00:16.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.105+0000 7f06177fe640 1 -- 172.21.15.178:0/3530779301 <== mon.2 v1:172.21.15.178:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v773) v1 ==== 221+0+0 (unknown 4226952807 0 0) 0x7f0610055780 con 0x7f0620108dc0 2024-02-14T18:00:16.107 INFO:tasks.workunit.client.0.smithi178.stderr:set-quota max_objects = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b 2024-02-14T18:00:16.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.108+0000 7f0628530640 1 -- 172.21.15.178:0/3530779301 >> v1:172.21.15.178:6800/4205070280 conn(0x7f060c06e8f0 legacy=0x7f060c070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:16.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.108+0000 7f0628530640 1 -- 172.21.15.178:0/3530779301 >> v1:172.21.15.178:6790/0 conn(0x7f0620108dc0 legacy=0x7f06201a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:16.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.109+0000 7f0628530640 1 -- 172.21.15.178:0/3530779301 shutdown_connections 2024-02-14T18:00:16.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.109+0000 7f0628530640 1 -- 172.21.15.178:0/3530779301 >> 172.21.15.178:0/3530779301 conn(0x7f0620075a30 msgr2=0x7f0620110050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:00:16.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.109+0000 7f0628530640 1 -- 172.21.15.178:0/3530779301 shutdown_connections 2024-02-14T18:00:16.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.109+0000 7f0628530640 1 -- 172.21.15.178:0/3530779301 wait complete. 2024-02-14T18:00:16.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ wait 151866 2024-02-14T18:00:16.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' 0 -ne 0 ']' 2024-02-14T18:00:16.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ true 2024-02-14T18:00:16.121 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put three /etc/passwd 2024-02-14T18:00:16.157 INFO:tasks.workunit.client.0.smithi178.stderr:++ uuidgen 2024-02-14T18:00:16.158 INFO:tasks.workunit.client.0.smithi178.stderr:+ pp=9abf3c01-ef88-436b-b12e-c5decaf64871 2024-02-14T18:00:16.158 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool create 9abf3c01-ef88-436b-b12e-c5decaf64871 12 2024-02-14T18:00:16.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.240+0000 7f55d15e2640 1 -- 172.21.15.178:0/2868040621 >> v1:172.21.15.178:6789/0 conn(0x7f55cc10da40 legacy=0x7f55cc10ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:16.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.240+0000 7f55d15e2640 1 -- 172.21.15.178:0/2868040621 shutdown_connections 2024-02-14T18:00:16.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.240+0000 7f55d15e2640 1 -- 172.21.15.178:0/2868040621 >> 172.21.15.178:0/2868040621 conn(0x7f55cc075a30 msgr2=0x7f55cc075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:00:16.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.240+0000 7f55d15e2640 1 -- 172.21.15.178:0/2868040621 shutdown_connections 2024-02-14T18:00:16.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.240+0000 7f55d15e2640 1 -- 172.21.15.178:0/2868040621 wait complete. 2024-02-14T18:00:16.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.241+0000 7f55d15e2640 1 Processor -- start 2024-02-14T18:00:16.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.241+0000 7f55d15e2640 1 -- start start 2024-02-14T18:00:16.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.241+0000 7f55d15e2640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f55cc1a7fe0 con 0x7f55cc109f00 2024-02-14T18:00:16.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.241+0000 7f55d15e2640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f55cc1a9180 con 0x7f55cc10da40 2024-02-14T18:00:16.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.241+0000 7f55d15e2640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f55cc1aa320 con 0x7f55cc0684d0 2024-02-14T18:00:16.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.241+0000 7f55c27fc640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f55cc109f00 0x7f55cc1a3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:60344/0 (socket says 172.21.15.178:60344) 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.241+0000 7f55caffd640 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f55cc0684d0 0x7f55cc1061a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:49632/0 (socket says 172.21.15.178:49632) 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.241+0000 7f55c27fc640 1 -- 172.21.15.178:0/402630933 learned_addr learned my addr 172.21.15.178:0/402630933 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.241+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174624243 0 0) 0x7f55cc1a7fe0 con 0x7f55cc109f00 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.241+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55b8003520 con 0x7f55cc109f00 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843736881 0 0) 0x7f55cc1aa320 con 0x7f55cc0684d0 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55cc1a7fe0 con 0x7f55cc0684d0 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294392606 0 0) 0x7f55cc1a9180 con 0x7f55cc10da40 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55cc1aa320 con 0x7f55cc10da40 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1102442002 0 0) 0x7f55b8003520 con 0x7f55cc109f00 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f55cc1a9180 con 0x7f55cc109f00 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f55bc001980 con 0x7f55cc109f00 2024-02-14T18:00:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 964330683 0 0) 0x7f55cc1a7fe0 con 0x7f55cc0684d0 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f55b8003520 con 0x7f55cc0684d0 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f55b4001850 con 0x7f55cc0684d0 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 316093918 0 0) 0x7f55cc1aa320 con 0x7f55cc10da40 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f55cc1a7fe0 con 0x7f55cc10da40 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f55c4001950 con 0x7f55cc10da40 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3080693298 0 0) 0x7f55cc1a9180 con 0x7f55cc109f00 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 >> v1:172.21.15.178:6790/0 conn(0x7f55cc0684d0 legacy=0x7f55cc1061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 >> v1:172.21.15.184:6789/0 conn(0x7f55cc10da40 legacy=0x7f55cc1a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc1ab4c0 con 0x7f55cc109f00 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55d15e2640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc1a81d0 con 0x7f55cc109f00 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55d15e2640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc1a8660 con 0x7f55cc109f00 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f55bc001c40 con 0x7f55cc109f00 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.242+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f55bc005c10 con 0x7f55cc109f00 2024-02-14T18:00:16.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.243+0000 7f55d15e2640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ac0050b0 con 0x7f55cc109f00 2024-02-14T18:00:16.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.243+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f55bc0038c0 con 0x7f55cc109f00 2024-02-14T18:00:16.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.244+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(773..773 src has 255..773) v4 ==== 7830+0+0 (unknown 2441385580 0 0) 0x7f55bc088f50 con 0x7f55cc109f00 2024-02-14T18:00:16.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.248+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f55bc0559a0 con 0x7f55cc109f00 2024-02-14T18:00:16.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:16.422+0000 7f55d15e2640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12} v 0) v1 -- 0x7f55ac0058e0 con 0x7f55cc109f00 2024-02-14T18:00:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:16 smithi184 ceph-mon[33536]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:00:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:16 smithi184 ceph-mon[33536]: osdmap e772: 8 total, 8 up, 8 in 2024-02-14T18:00:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:16 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3530779301' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:16 smithi184 ceph-mon[33536]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:16 smithi184 ceph-mon[33536]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' no longer out of quota; removing NO_QUOTA flag 2024-02-14T18:00:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:16 smithi184 ceph-mon[33536]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:00:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:16 smithi184 ceph-mon[33536]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:00:16.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:16 smithi184 ceph-mon[33536]: osdmap e773: 8 total, 8 up, 8 in 2024-02-14T18:00:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[39811]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:00:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[39811]: osdmap e772: 8 total, 8 up, 8 in 2024-02-14T18:00:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3530779301' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[39811]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[39811]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' no longer out of quota; removing NO_QUOTA flag 2024-02-14T18:00:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[39811]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:00:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[39811]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:00:17.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[39811]: osdmap e773: 8 total, 8 up, 8 in 2024-02-14T18:00:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[30321]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:00:17.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[30321]: osdmap e772: 8 total, 8 up, 8 in 2024-02-14T18:00:17.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3530779301' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:17.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[30321]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:00:17.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[30321]: pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' no longer out of quota; removing NO_QUOTA flag 2024-02-14T18:00:17.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[30321]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:00:17.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[30321]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:00:17.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:16 smithi178 ceph-mon[30321]: osdmap e773: 8 total, 8 up, 8 in 2024-02-14T18:00:17.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.104+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12}]=0 pool '9abf3c01-ef88-436b-b12e-c5decaf64871' created v774) v1 ==== 176+0+0 (unknown 1889387591 0 0) 0x7f55bc053fa0 con 0x7f55cc109f00 2024-02-14T18:00:17.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.215+0000 7f55d15e2640 1 -- 172.21.15.178:0/402630933 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12} v 0) v1 -- 0x7f55ac005aa0 con 0x7f55cc109f00 2024-02-14T18:00:17.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.216+0000 7f55c8ff9640 1 -- 172.21.15.178:0/402630933 <== mon.0 v1:172.21.15.178:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12}]=0 pool '9abf3c01-ef88-436b-b12e-c5decaf64871' already exists v774) v1 ==== 183+0+0 (unknown 1776970931 0 0) 0x7f55bc0557c0 con 0x7f55cc109f00 2024-02-14T18:00:17.217 INFO:tasks.workunit.client.0.smithi178.stderr:pool '9abf3c01-ef88-436b-b12e-c5decaf64871' already exists 2024-02-14T18:00:17.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.218+0000 7f55d15e2640 1 -- 172.21.15.178:0/402630933 >> v1:172.21.15.178:6800/4205070280 conn(0x7f55b806ea30 legacy=0x7f55b8070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:17.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.218+0000 7f55d15e2640 1 -- 172.21.15.178:0/402630933 >> v1:172.21.15.178:6789/0 conn(0x7f55cc109f00 legacy=0x7f55cc1a3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:17.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.218+0000 7f55d15e2640 1 -- 172.21.15.178:0/402630933 shutdown_connections 2024-02-14T18:00:17.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.218+0000 7f55d15e2640 1 -- 172.21.15.178:0/402630933 >> 172.21.15.178:0/402630933 conn(0x7f55cc075a30 msgr2=0x7f55cc110df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:00:17.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.218+0000 7f55d15e2640 1 -- 172.21.15.178:0/402630933 shutdown_connections 2024-02-14T18:00:17.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.218+0000 7f55d15e2640 1 -- 172.21.15.178:0/402630933 wait complete. 2024-02-14T18:00:17.228 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool application enable 9abf3c01-ef88-436b-b12e-c5decaf64871 rados 2024-02-14T18:00:17.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.308+0000 7f6d17d13640 1 -- 172.21.15.178:0/34996361 >> v1:172.21.15.178:6790/0 conn(0x7f6d10100140 legacy=0x7f6d10100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:17.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d17d13640 1 -- 172.21.15.178:0/34996361 shutdown_connections 2024-02-14T18:00:17.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d17d13640 1 -- 172.21.15.178:0/34996361 >> 172.21.15.178:0/34996361 conn(0x7f6d100fb980 msgr2=0x7f6d100fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:00:17.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d17d13640 1 -- 172.21.15.178:0/34996361 shutdown_connections 2024-02-14T18:00:17.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d17d13640 1 -- 172.21.15.178:0/34996361 wait complete. 2024-02-14T18:00:17.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d17d13640 1 Processor -- start 2024-02-14T18:00:17.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d17d13640 1 -- start start 2024-02-14T18:00:17.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d17d13640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6d101a5ec0 con 0x7f6d10100140 2024-02-14T18:00:17.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d17d13640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6d101a70c0 con 0x7f6d10108dc0 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d17d13640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6d101a82c0 con 0x7f6d1010c930 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d15aaf640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f6d10100140 0x7f6d10069f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:60372/0 (socket says 172.21.15.178:60372) 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d162b0640 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f6d1010c930 0x7f6d101a45c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:49656/0 (socket says 172.21.15.178:49656) 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.309+0000 7f6d15aaf640 1 -- 172.21.15.178:0/3878762433 learned_addr learned my addr 172.21.15.178:0/3878762433 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037697956 0 0) 0x7f6d101a5ec0 con 0x7f6d10100140 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cfc003160 con 0x7f6d10100140 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965104196 0 0) 0x7f6d101a82c0 con 0x7f6d1010c930 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d101a5ec0 con 0x7f6d1010c930 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276930123 0 0) 0x7f6d101a70c0 con 0x7f6d10108dc0 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d101a82c0 con 0x7f6d10108dc0 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2592974216 0 0) 0x7f6cfc003160 con 0x7f6d10100140 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d101a70c0 con 0x7f6d10100140 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f6d0c0044b0 con 0x7f6d10100140 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 194463562 0 0) 0x7f6d101a5ec0 con 0x7f6d1010c930 2024-02-14T18:00:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6cfc003160 con 0x7f6d1010c930 2024-02-14T18:00:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f6d08001b30 con 0x7f6d1010c930 2024-02-14T18:00:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2283327358 0 0) 0x7f6d101a70c0 con 0x7f6d10100140 2024-02-14T18:00:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 >> v1:172.21.15.178:6790/0 conn(0x7f6d1010c930 legacy=0x7f6d101a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 >> v1:172.21.15.184:6789/0 conn(0x7f6d10108dc0 legacy=0x7f6d101a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d101a94c0 con 0x7f6d10100140 2024-02-14T18:00:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d17d13640 1 -- 172.21.15.178:0/3878762433 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d101a8450 con 0x7f6d10100140 2024-02-14T18:00:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d17d13640 1 -- 172.21.15.178:0/3878762433 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d101a88e0 con 0x7f6d10100140 2024-02-14T18:00:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f6d0c003b10 con 0x7f6d10100140 2024-02-14T18:00:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.310+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f6d0c004ed0 con 0x7f6d10100140 2024-02-14T18:00:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.311+0000 7f6d17d13640 1 -- 172.21.15.178:0/3878762433 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf4005080 con 0x7f6d10100140 2024-02-14T18:00:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.312+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f6d0c003680 con 0x7f6d10100140 2024-02-14T18:00:17.316 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.312+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(774..774 src has 255..774) v4 ==== 8205+0+0 (unknown 1051867261 0 0) 0x7f6d0c089390 con 0x7f6d10100140 2024-02-14T18:00:17.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.317+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f6d0c055e90 con 0x7f6d10100140 2024-02-14T18:00:17.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:17.494+0000 7f6d17d13640 1 -- 172.21.15.178:0/3878762433 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"} v 0) v1 -- 0x7f6cf4005330 con 0x7f6d10100140 2024-02-14T18:00:17.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/402630933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12}]: dispatch 2024-02-14T18:00:17.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:17 smithi184 ceph-mon[33536]: pgmap v1231: 176 pgs: 176 active+clean; 720 KiB data, 831 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:17.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/402630933' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12}]': finished 2024-02-14T18:00:17.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:17 smithi184 ceph-mon[33536]: osdmap e774: 8 total, 8 up, 8 in 2024-02-14T18:00:17.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/402630933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12}]: dispatch 2024-02-14T18:00:17.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:17 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]: dispatch 2024-02-14T18:00:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/402630933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12}]: dispatch 2024-02-14T18:00:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[39811]: pgmap v1231: 176 pgs: 176 active+clean; 720 KiB data, 831 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/402630933' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12}]': finished 2024-02-14T18:00:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[39811]: osdmap e774: 8 total, 8 up, 8 in 2024-02-14T18:00:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/402630933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12}]: dispatch 2024-02-14T18:00:18.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]: dispatch 2024-02-14T18:00:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/402630933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12}]: dispatch 2024-02-14T18:00:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[30321]: pgmap v1231: 176 pgs: 176 active+clean; 720 KiB data, 831 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/402630933' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12}]': finished 2024-02-14T18:00:18.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[30321]: osdmap e774: 8 total, 8 up, 8 in 2024-02-14T18:00:18.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/402630933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pg_num": 12}]: dispatch 2024-02-14T18:00:18.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:17 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]: dispatch 2024-02-14T18:00:18.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:18.107+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]=0 enabled application 'rados' on pool '9abf3c01-ef88-436b-b12e-c5decaf64871' v775) v1 ==== 213+0+0 (unknown 687191150 0 0) 0x7f6d0c054200 con 0x7f6d10100140 2024-02-14T18:00:18.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:18.225+0000 7f6d17d13640 1 -- 172.21.15.178:0/3878762433 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"} v 0) v1 -- 0x7f6cf4001090 con 0x7f6d10100140 2024-02-14T18:00:19.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.112+0000 7f6d06ffd640 1 -- 172.21.15.178:0/3878762433 <== mon.0 v1:172.21.15.178:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]=0 enabled application 'rados' on pool '9abf3c01-ef88-436b-b12e-c5decaf64871' v776) v1 ==== 213+0+0 (unknown 1525921053 0 0) 0x7f6d0c055a20 con 0x7f6d10100140 2024-02-14T18:00:19.114 INFO:tasks.workunit.client.0.smithi178.stderr:enabled application 'rados' on pool '9abf3c01-ef88-436b-b12e-c5decaf64871' 2024-02-14T18:00:19.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.116+0000 7f6d17d13640 1 -- 172.21.15.178:0/3878762433 >> v1:172.21.15.178:6800/4205070280 conn(0x7f6cfc06e720 legacy=0x7f6cfc070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:19.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.116+0000 7f6d17d13640 1 -- 172.21.15.178:0/3878762433 >> v1:172.21.15.178:6789/0 conn(0x7f6d10100140 legacy=0x7f6d10069f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:19.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.117+0000 7f6d17d13640 1 -- 172.21.15.178:0/3878762433 shutdown_connections 2024-02-14T18:00:19.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.117+0000 7f6d17d13640 1 -- 172.21.15.178:0/3878762433 >> 172.21.15.178:0/3878762433 conn(0x7f6d100fb980 msgr2=0x7f6d1010f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:00:19.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.117+0000 7f6d17d13640 1 -- 172.21.15.178:0/3878762433 shutdown_connections 2024-02-14T18:00:19.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.117+0000 7f6d17d13640 1 -- 172.21.15.178:0/3878762433 wait complete. 2024-02-14T18:00:19.129 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool set-quota 9abf3c01-ef88-436b-b12e-c5decaf64871 max_objects 10 2024-02-14T18:00:19.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.210+0000 7f141a9bc640 1 -- 172.21.15.178:0/1054133807 >> v1:172.21.15.178:6789/0 conn(0x7f1414109ed0 legacy=0x7f141410c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:19.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.210+0000 7f141a9bc640 1 -- 172.21.15.178:0/1054133807 shutdown_connections 2024-02-14T18:00:19.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.210+0000 7f141a9bc640 1 -- 172.21.15.178:0/1054133807 >> 172.21.15.178:0/1054133807 conn(0x7f14140fcaf0 msgr2=0x7f14140fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:00:19.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.210+0000 7f141a9bc640 1 -- 172.21.15.178:0/1054133807 shutdown_connections 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.210+0000 7f141a9bc640 1 -- 172.21.15.178:0/1054133807 wait complete. 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.210+0000 7f141a9bc640 1 Processor -- start 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.210+0000 7f141a9bc640 1 -- start start 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f141a9bc640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f14141a5e40 con 0x7f1414109ed0 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f141a9bc640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f14141a7040 con 0x7f1414101250 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f141a9bc640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f14141a8240 con 0x7f141410da40 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14137fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f1414109ed0 0x7f14141a0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:60390/0 (socket says 172.21.15.178:60390) 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f1418f59640 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f141410da40 0x7f14141a4540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:49680/0 (socket says 172.21.15.178:49680) 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14137fe640 1 -- 172.21.15.178:0/781042656 learned_addr learned my addr 172.21.15.178:0/781042656 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261671268 0 0) 0x7f14141a5e40 con 0x7f1414109ed0 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13fc004950 con 0x7f1414109ed0 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661093248 0 0) 0x7f14141a8240 con 0x7f141410da40 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14141a5e40 con 0x7f141410da40 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964054521 0 0) 0x7f14141a7040 con 0x7f1414101250 2024-02-14T18:00:19.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14141a8240 con 0x7f1414101250 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 659571673 0 0) 0x7f13fc004950 con 0x7f1414109ed0 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f14141a7040 con 0x7f1414109ed0 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f14080014f0 con 0x7f1414109ed0 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2533985903 0 0) 0x7f14141a5e40 con 0x7f141410da40 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13fc004950 con 0x7f141410da40 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f140c001bd0 con 0x7f141410da40 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1717979019 0 0) 0x7f14141a8240 con 0x7f1414101250 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f14141a5e40 con 0x7f1414101250 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.211+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f1404001870 con 0x7f1414101250 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.212+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3742062197 0 0) 0x7f14141a7040 con 0x7f1414109ed0 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.212+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 >> v1:172.21.15.178:6790/0 conn(0x7f141410da40 legacy=0x7f14141a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.212+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 >> v1:172.21.15.184:6789/0 conn(0x7f1414101250 legacy=0x7f1414106460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.212+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14141a9440 con 0x7f1414109ed0 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.212+0000 7f141a9bc640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14141a8430 con 0x7f1414109ed0 2024-02-14T18:00:19.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.212+0000 7f141a9bc640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f14141a88c0 con 0x7f1414109ed0 2024-02-14T18:00:19.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.212+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f1408001ad0 con 0x7f1414109ed0 2024-02-14T18:00:19.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.212+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f1408004ee0 con 0x7f1414109ed0 2024-02-14T18:00:19.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.213+0000 7f141a9bc640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1400005080 con 0x7f1414109ed0 2024-02-14T18:00:19.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.213+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f14080050e0 con 0x7f1414109ed0 2024-02-14T18:00:19.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.213+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(776..776 src has 255..776) v4 ==== 8218+0+0 (unknown 1284851995 0 0) 0x7f1408089310 con 0x7f1414109ed0 2024-02-14T18:00:19.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.218+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f1408055e20 con 0x7f1414109ed0 2024-02-14T18:00:19.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:19.391+0000 7f141a9bc640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f1400005330 con 0x7f1414109ed0 2024-02-14T18:00:19.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]': finished 2024-02-14T18:00:19.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:19 smithi184 ceph-mon[33536]: osdmap e775: 8 total, 8 up, 8 in 2024-02-14T18:00:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:19 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]: dispatch 2024-02-14T18:00:19.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:19 smithi184 ceph-mon[33536]: pgmap v1234: 188 pgs: 12 unknown, 176 active+clean; 720 KiB data, 831 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-14T18:00:19.582 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]': finished 2024-02-14T18:00:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:19 smithi178 ceph-mon[30321]: osdmap e775: 8 total, 8 up, 8 in 2024-02-14T18:00:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:19 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]: dispatch 2024-02-14T18:00:19.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:19 smithi178 ceph-mon[30321]: pgmap v1234: 188 pgs: 12 unknown, 176 active+clean; 720 KiB data, 831 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-14T18:00:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]': finished 2024-02-14T18:00:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:19 smithi178 ceph-mon[39811]: osdmap e775: 8 total, 8 up, 8 in 2024-02-14T18:00:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:19 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]: dispatch 2024-02-14T18:00:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:19 smithi178 ceph-mon[39811]: pgmap v1234: 188 pgs: 12 unknown, 176 active+clean; 720 KiB data, 831 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-14T18:00:20.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:20.119+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 9abf3c01-ef88-436b-b12e-c5decaf64871 v777) v1 ==== 223+0+0 (unknown 2839828831 0 0) 0x7f1408054190 con 0x7f1414109ed0 2024-02-14T18:00:20.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:20.230+0000 7f141a9bc640 1 -- 172.21.15.178:0/781042656 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f1400001090 con 0x7f1414109ed0 2024-02-14T18:00:20.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]': finished 2024-02-14T18:00:20.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:20 smithi184 ceph-mon[33536]: osdmap e776: 8 total, 8 up, 8 in 2024-02-14T18:00:20.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:20 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:00:20.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]': finished 2024-02-14T18:00:20.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:20 smithi178 ceph-mon[39811]: osdmap e776: 8 total, 8 up, 8 in 2024-02-14T18:00:20.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:20 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:00:20.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3878762433' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "app": "rados"}]': finished 2024-02-14T18:00:20.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:20 smithi178 ceph-mon[30321]: osdmap e776: 8 total, 8 up, 8 in 2024-02-14T18:00:20.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:20 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:00:21.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:21.124+0000 7f14117fa640 1 -- 172.21.15.178:0/781042656 <== mon.0 v1:172.21.15.178:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 9abf3c01-ef88-436b-b12e-c5decaf64871 v778) v1 ==== 223+0+0 (unknown 1921693391 0 0) 0x7f14080559b0 con 0x7f1414109ed0 2024-02-14T18:00:21.126 INFO:tasks.workunit.client.0.smithi178.stderr:set-quota max_objects = 10 for pool 9abf3c01-ef88-436b-b12e-c5decaf64871 2024-02-14T18:00:21.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:21.128+0000 7f141a9bc640 1 -- 172.21.15.178:0/781042656 >> v1:172.21.15.178:6800/4205070280 conn(0x7f13fc06e990 legacy=0x7f13fc070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:21.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:21.129+0000 7f141a9bc640 1 -- 172.21.15.178:0/781042656 >> v1:172.21.15.178:6789/0 conn(0x7f1414109ed0 legacy=0x7f14141a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:00:21.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:21.129+0000 7f141a9bc640 1 -- 172.21.15.178:0/781042656 shutdown_connections 2024-02-14T18:00:21.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:21.129+0000 7f141a9bc640 1 -- 172.21.15.178:0/781042656 >> 172.21.15.178:0/781042656 conn(0x7f14140fcaf0 msgr2=0x7f1414110f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:00:21.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:21.129+0000 7f141a9bc640 1 -- 172.21.15.178:0/781042656 shutdown_connections 2024-02-14T18:00:21.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:00:21.129+0000 7f141a9bc640 1 -- 172.21.15.178:0/781042656 wait complete. 2024-02-14T18:00:21.142 INFO:tasks.workunit.client.0.smithi178.stderr:+ sleep 30 2024-02-14T18:00:21.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:00:21.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:21 smithi184 ceph-mon[33536]: osdmap e777: 8 total, 8 up, 8 in 2024-02-14T18:00:21.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:00:21.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:21 smithi184 ceph-mon[33536]: pgmap v1237: 188 pgs: 9 unknown, 179 active+clean; 720 KiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:21.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:00:21.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:21 smithi178 ceph-mon[39811]: osdmap e777: 8 total, 8 up, 8 in 2024-02-14T18:00:21.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:00:21.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:21 smithi178 ceph-mon[39811]: pgmap v1237: 188 pgs: 9 unknown, 179 active+clean; 720 KiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:00:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:21 smithi178 ceph-mon[30321]: osdmap e777: 8 total, 8 up, 8 in 2024-02-14T18:00:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:00:21.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:21 smithi178 ceph-mon[30321]: pgmap v1237: 188 pgs: 9 unknown, 179 active+clean; 720 KiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:22.127 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:00:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:00:22.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:00:22.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:22 smithi184 ceph-mon[33536]: osdmap e778: 8 total, 8 up, 8 in 2024-02-14T18:00:22.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:00:22.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:22 smithi178 ceph-mon[39811]: osdmap e778: 8 total, 8 up, 8 in 2024-02-14T18:00:22.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/781042656' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:00:22.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:22 smithi178 ceph-mon[30321]: osdmap e778: 8 total, 8 up, 8 in 2024-02-14T18:00:23.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:23 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:23.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:23 smithi184 ceph-mon[33536]: pgmap v1239: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.5 KiB/s wr, 1 op/s 2024-02-14T18:00:23.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:23 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:23.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:23 smithi178 ceph-mon[39811]: pgmap v1239: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.5 KiB/s wr, 1 op/s 2024-02-14T18:00:23.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:23 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:23.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:23 smithi178 ceph-mon[30321]: pgmap v1239: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.5 KiB/s wr, 1 op/s 2024-02-14T18:00:24.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:24 smithi178 ceph-mon[30321]: pgmap v1240: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:00:24.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:24 smithi178 ceph-mon[39811]: pgmap v1240: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:00:24.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:24 smithi184 ceph-mon[33536]: pgmap v1240: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:00:26.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:00:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:00:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:27.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:27 smithi184 ceph-mon[33536]: pgmap v1241: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-02-14T18:00:27.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:27 smithi178 ceph-mon[39811]: pgmap v1241: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-02-14T18:00:27.583 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:27 smithi178 ceph-mon[30321]: pgmap v1241: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-02-14T18:00:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:28 smithi178 ceph-mon[39811]: pgmap v1242: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-02-14T18:00:28.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:28 smithi178 ceph-mon[30321]: pgmap v1242: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-02-14T18:00:28.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:28 smithi184 ceph-mon[33536]: pgmap v1242: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-02-14T18:00:30.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:29 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:00:30.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:29 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:30.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:29 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:00:30.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:29 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:30.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:29 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:00:30.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:29 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:31.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:30 smithi184 ceph-mon[33536]: pgmap v1243: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-14T18:00:31.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:30 smithi178 ceph-mon[39811]: pgmap v1243: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-14T18:00:31.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:30 smithi178 ceph-mon[30321]: pgmap v1243: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-14T18:00:32.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:00:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:00:32.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:32 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:32.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:32 smithi178 ceph-mon[39811]: pgmap v1244: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-02-14T18:00:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:32 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:32.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:32 smithi178 ceph-mon[30321]: pgmap v1244: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-02-14T18:00:32.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:32 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:32.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:32 smithi184 ceph-mon[33536]: pgmap v1244: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-02-14T18:00:34.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:34 smithi178 ceph-mon[39811]: pgmap v1245: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:34.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:34 smithi178 ceph-mon[30321]: pgmap v1245: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:34.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:34 smithi184 ceph-mon[33536]: pgmap v1245: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:36.082 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:00:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:00:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:37.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:37 smithi178 ceph-mon[39811]: pgmap v1246: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:37.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:37 smithi178 ceph-mon[30321]: pgmap v1246: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:37.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:37 smithi184 ceph-mon[33536]: pgmap v1246: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:39.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:39 smithi178 ceph-mon[39811]: pgmap v1247: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:39.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:39 smithi178 ceph-mon[30321]: pgmap v1247: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:39.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:39 smithi184 ceph-mon[33536]: pgmap v1247: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:41.764 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:41 smithi184 ceph-mon[33536]: pgmap v1248: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:41.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:41 smithi178 ceph-mon[39811]: pgmap v1248: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:41.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:41 smithi178 ceph-mon[30321]: pgmap v1248: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:42.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:00:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:00:42.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:42 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:42.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:42 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:42.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:42 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:43.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:43 smithi178 ceph-mon[39811]: pgmap v1249: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:43.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:43 smithi178 ceph-mon[30321]: pgmap v1249: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:43.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:43 smithi184 ceph-mon[33536]: pgmap v1249: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:44.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:44 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:44.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:44 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:44.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:44 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:45.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:45 smithi178 ceph-mon[39811]: pgmap v1250: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:45.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:45 smithi178 ceph-mon[30321]: pgmap v1250: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:45.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:00:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:00:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:45.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:45 smithi184 ceph-mon[33536]: pgmap v1250: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:46.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:46 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:00:46.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:46 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:00:46.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:46 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:00:47.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:47 smithi178 ceph-mon[39811]: pgmap v1251: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:47 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:00:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:47 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:00:47.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:47 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:00:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:47 smithi178 ceph-mon[30321]: pgmap v1251: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:47 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:00:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:47 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:00:47.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:47 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:00:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:47 smithi184 ceph-mon[33536]: pgmap v1251: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:47 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:00:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:47 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:00:47.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:47 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:00:49.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:49 smithi178 ceph-mon[39811]: pgmap v1252: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:49.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:49 smithi178 ceph-mon[30321]: pgmap v1252: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:49.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:49 smithi184 ceph-mon[33536]: pgmap v1252: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:00:51.144 INFO:tasks.workunit.client.0.smithi178.stderr:++ seq 1 10 2024-02-14T18:00:51.144 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T18:00:51.145 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p 9abf3c01-ef88-436b-b12e-c5decaf64871 put obj1 /etc/passwd 2024-02-14T18:00:51.184 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T18:00:51.184 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p 9abf3c01-ef88-436b-b12e-c5decaf64871 put obj2 /etc/passwd 2024-02-14T18:00:51.219 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T18:00:51.219 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p 9abf3c01-ef88-436b-b12e-c5decaf64871 put obj3 /etc/passwd 2024-02-14T18:00:51.253 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T18:00:51.253 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p 9abf3c01-ef88-436b-b12e-c5decaf64871 put obj4 /etc/passwd 2024-02-14T18:00:51.287 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T18:00:51.287 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p 9abf3c01-ef88-436b-b12e-c5decaf64871 put obj5 /etc/passwd 2024-02-14T18:00:51.319 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T18:00:51.320 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p 9abf3c01-ef88-436b-b12e-c5decaf64871 put obj6 /etc/passwd 2024-02-14T18:00:51.352 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T18:00:51.352 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p 9abf3c01-ef88-436b-b12e-c5decaf64871 put obj7 /etc/passwd 2024-02-14T18:00:51.384 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T18:00:51.384 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p 9abf3c01-ef88-436b-b12e-c5decaf64871 put obj8 /etc/passwd 2024-02-14T18:00:51.416 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T18:00:51.416 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p 9abf3c01-ef88-436b-b12e-c5decaf64871 put obj9 /etc/passwd 2024-02-14T18:00:51.449 INFO:tasks.workunit.client.0.smithi178.stderr:+ for f in `seq 1 10` 2024-02-14T18:00:51.449 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p 9abf3c01-ef88-436b-b12e-c5decaf64871 put obj10 /etc/passwd 2024-02-14T18:00:51.481 INFO:tasks.workunit.client.0.smithi178.stderr:+ sleep 30 2024-02-14T18:00:51.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:51 smithi178 ceph-mon[39811]: pgmap v1253: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:51.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:51 smithi178 ceph-mon[30321]: pgmap v1253: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:51.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:51 smithi184 ceph-mon[33536]: pgmap v1253: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:51.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:00:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:00:52.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:52 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:52.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:52 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:52.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:52 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:53.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:53 smithi178 ceph-mon[39811]: pgmap v1254: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:53.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:53 smithi178 ceph-mon[30321]: pgmap v1254: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:53.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:53 smithi184 ceph-mon[33536]: pgmap v1254: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:55.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:55 smithi178 ceph-mon[39811]: pgmap v1255: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:55.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:55 smithi178 ceph-mon[30321]: pgmap v1255: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:55.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:00:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:00:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:55.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:55 smithi184 ceph-mon[33536]: pgmap v1255: 188 pgs: 188 active+clean; 743 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:00:57.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:57 smithi178 ceph-mon[39811]: pgmap v1256: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-14T18:00:57.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:57 smithi178 ceph-mon[30321]: pgmap v1256: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-14T18:00:57.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:57 smithi184 ceph-mon[33536]: pgmap v1256: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-14T18:00:59.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:59 smithi178 ceph-mon[39811]: pgmap v1257: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-14T18:00:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:00:59 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:59 smithi178 ceph-mon[30321]: pgmap v1257: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-14T18:00:59.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:00:59 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:59.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:59 smithi184 ceph-mon[33536]: pgmap v1257: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-14T18:00:59.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:00:59 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:01.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:01 smithi178 ceph-mon[39811]: pgmap v1258: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-14T18:01:01.794 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:01 smithi178 ceph-mon[39811]: pool '9abf3c01-ef88-436b-b12e-c5decaf64871' is full (reached quota's max_objects: 10) 2024-02-14T18:01:01.794 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:01 smithi178 ceph-mon[39811]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:01:01.794 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:01 smithi178 ceph-mon[39811]: osdmap e779: 8 total, 8 up, 8 in 2024-02-14T18:01:01.794 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:01 smithi178 ceph-mon[30321]: pgmap v1258: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-14T18:01:01.794 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:01 smithi178 ceph-mon[30321]: pool '9abf3c01-ef88-436b-b12e-c5decaf64871' is full (reached quota's max_objects: 10) 2024-02-14T18:01:01.794 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:01 smithi178 ceph-mon[30321]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:01:01.794 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:01 smithi178 ceph-mon[30321]: osdmap e779: 8 total, 8 up, 8 in 2024-02-14T18:01:01.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:01 smithi184 ceph-mon[33536]: pgmap v1258: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-14T18:01:01.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:01 smithi184 ceph-mon[33536]: pool '9abf3c01-ef88-436b-b12e-c5decaf64871' is full (reached quota's max_objects: 10) 2024-02-14T18:01:01.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:01 smithi184 ceph-mon[33536]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:01:01.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:01 smithi184 ceph-mon[33536]: osdmap e779: 8 total, 8 up, 8 in 2024-02-14T18:01:01.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:01:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:01:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:02 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:02.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:02 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:02.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:02 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:03.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:03 smithi178 ceph-mon[39811]: pgmap v1260: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:01:03.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:03 smithi178 ceph-mon[30321]: pgmap v1260: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:01:03.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:03 smithi184 ceph-mon[33536]: pgmap v1260: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:01:05.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:05 smithi178 ceph-mon[39811]: pgmap v1261: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:01:05.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:05 smithi178 ceph-mon[30321]: pgmap v1261: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:01:05.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:05 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:01:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:05.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:05 smithi184 ceph-mon[33536]: pgmap v1261: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:01:07.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:07 smithi178 ceph-mon[39811]: pgmap v1262: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:07.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:07 smithi178 ceph-mon[30321]: pgmap v1262: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:07.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:07 smithi184 ceph-mon[33536]: pgmap v1262: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:09 smithi178 ceph-mon[39811]: pgmap v1263: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:09.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:09 smithi178 ceph-mon[30321]: pgmap v1263: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:09.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:09 smithi184 ceph-mon[33536]: pgmap v1263: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:11.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:11 smithi178 ceph-mon[39811]: pgmap v1264: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:11.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:11 smithi178 ceph-mon[30321]: pgmap v1264: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:11.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:11 smithi184 ceph-mon[33536]: pgmap v1264: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:11.969 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:01:11 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:01:12.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:12 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:12.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:12 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:12.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:12 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:13.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:13 smithi178 ceph-mon[39811]: pgmap v1265: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-02-14T18:01:13.833 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:13 smithi178 ceph-mon[30321]: pgmap v1265: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-02-14T18:01:13.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:13 smithi184 ceph-mon[33536]: pgmap v1265: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 902 B/s rd, 0 op/s 2024-02-14T18:01:15.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:14 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:01:15.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:14 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:14 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:01:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:14 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:14 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:01:15.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:14 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:15.904 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:15 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:01:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:16.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:15 smithi184 ceph-mon[33536]: pgmap v1266: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:16.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:15 smithi178 ceph-mon[39811]: pgmap v1266: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:16.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:15 smithi178 ceph-mon[30321]: pgmap v1266: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:18.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:17 smithi184 ceph-mon[33536]: pgmap v1267: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:18.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:17 smithi178 ceph-mon[30321]: pgmap v1267: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:18.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:17 smithi178 ceph-mon[39811]: pgmap v1267: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:20.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:19 smithi184 ceph-mon[33536]: pgmap v1268: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:20.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:19 smithi178 ceph-mon[39811]: pgmap v1268: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:20.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:19 smithi178 ceph-mon[30321]: pgmap v1268: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:21.482 INFO:tasks.workunit.client.0.smithi178.stderr:+ rados -p c8dfcd28-16bd-46b5-a2ee-f0743e289c3b put threemore /etc/passwd 2024-02-14T18:01:21.522 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool set-quota c8dfcd28-16bd-46b5-a2ee-f0743e289c3b max_bytes 0 2024-02-14T18:01:21.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.610+0000 7f0abcf3e640 1 -- 172.21.15.178:0/1312585895 >> v1:172.21.15.178:6790/0 conn(0x7f0ab8109eb0 legacy=0x7f0ab810c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:21.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.610+0000 7f0abcf3e640 1 -- 172.21.15.178:0/1312585895 shutdown_connections 2024-02-14T18:01:21.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.610+0000 7f0abcf3e640 1 -- 172.21.15.178:0/1312585895 >> 172.21.15.178:0/1312585895 conn(0x7f0ab80fca70 msgr2=0x7f0ab80fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:01:21.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.610+0000 7f0abcf3e640 1 -- 172.21.15.178:0/1312585895 shutdown_connections 2024-02-14T18:01:21.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.610+0000 7f0abcf3e640 1 -- 172.21.15.178:0/1312585895 wait complete. 2024-02-14T18:01:21.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.610+0000 7f0abcf3e640 1 Processor -- start 2024-02-14T18:01:21.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.610+0000 7f0abcf3e640 1 -- start start 2024-02-14T18:01:21.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0abcf3e640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0ab81a5dc0 con 0x7f0ab8101230 2024-02-14T18:01:21.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0abcf3e640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0ab81a6fc0 con 0x7f0ab8109eb0 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0abcf3e640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0ab81a81c0 con 0x7f0ab810da20 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0ab659c640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f0ab8101230 0x7f0ab81063e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:35332/0 (socket says 172.21.15.178:35332) 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0ab6d9d640 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f0ab810da20 0x7f0ab81a44c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:46620/0 (socket says 172.21.15.178:46620) 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0ab659c640 1 -- 172.21.15.178:0/595415330 learned_addr learned my addr 172.21.15.178:0/595415330 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295604479 0 0) 0x7f0ab81a81c0 con 0x7f0ab810da20 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa4003160 con 0x7f0ab810da20 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494372184 0 0) 0x7f0ab81a5dc0 con 0x7f0ab8101230 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab81a81c0 con 0x7f0ab8101230 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31746918 0 0) 0x7f0ab81a6fc0 con 0x7f0ab8109eb0 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab81a5dc0 con 0x7f0ab8109eb0 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 861211835 0 0) 0x7f0aa4003160 con 0x7f0ab810da20 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0ab81a6fc0 con 0x7f0ab810da20 2024-02-14T18:01:21.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f0ab0001bb0 con 0x7f0ab810da20 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1820756170 0 0) 0x7f0ab81a81c0 con 0x7f0ab8101230 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0aa4003160 con 0x7f0ab8101230 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f0aa80019f0 con 0x7f0ab8101230 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2039461689 0 0) 0x7f0ab81a5dc0 con 0x7f0ab8109eb0 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0ab81a81c0 con 0x7f0ab8109eb0 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f0aa0004370 con 0x7f0ab8109eb0 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3172008909 0 0) 0x7f0ab81a6fc0 con 0x7f0ab810da20 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 >> v1:172.21.15.184:6789/0 conn(0x7f0ab8109eb0 legacy=0x7f0ab81a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 >> v1:172.21.15.178:6789/0 conn(0x7f0ab8101230 legacy=0x7f0ab81063e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab81a93c0 con 0x7f0ab810da20 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.611+0000 7f0abcf3e640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab81a5fb0 con 0x7f0ab810da20 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.612+0000 7f0abcf3e640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab81a6440 con 0x7f0ab810da20 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.612+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f0ab0003e20 con 0x7f0ab810da20 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.612+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f0ab0005eb0 con 0x7f0ab810da20 2024-02-14T18:01:21.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.613+0000 7f0abcf3e640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a98005080 con 0x7f0ab810da20 2024-02-14T18:01:21.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.613+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f0ab0003990 con 0x7f0ab810da20 2024-02-14T18:01:21.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.613+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(779..779 src has 255..779) v4 ==== 8218+0+0 (unknown 1326886092 0 0) 0x7f0ab0089490 con 0x7f0ab810da20 2024-02-14T18:01:21.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.614+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=780}) v3 -- 0x7f0ab81a6fc0 con 0x7f0ab810da20 2024-02-14T18:01:21.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.618+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f0ab0055f80 con 0x7f0ab810da20 2024-02-14T18:01:21.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.793+0000 7f0abcf3e640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f0a98005330 con 0x7f0ab810da20 2024-02-14T18:01:21.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.911+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 10 ==== osd_map(780..780 src has 255..780) v4 ==== 628+0+0 (unknown 2880714347 0 0) 0x7f0ab00568b0 con 0x7f0ab810da20 2024-02-14T18:01:21.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.911+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=781}) v3 -- 0x7f0ab81a5dc0 con 0x7f0ab810da20 2024-02-14T18:01:21.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:21.914+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v780) v1 ==== 217+0+0 (unknown 1835646836 0 0) 0x7f0ab00542f0 con 0x7f0ab810da20 2024-02-14T18:01:22.023 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:22.022+0000 7f0abcf3e640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f0a98000c40 con 0x7f0ab810da20 2024-02-14T18:01:22.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:01:21 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:01:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:21 smithi184 ceph-mon[33536]: pgmap v1269: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:21 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/595415330' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:22.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:21 smithi184 ceph-mon[33536]: from='client.57254 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:22.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:21 smithi178 ceph-mon[39811]: pgmap v1269: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:22.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:21 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/595415330' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:22.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:21 smithi178 ceph-mon[39811]: from='client.57254 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:21 smithi178 ceph-mon[30321]: pgmap v1269: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:21 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/595415330' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:22.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:21 smithi178 ceph-mon[30321]: from='client.57254 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:22.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:22.924+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 12 ==== osd_map(781..781 src has 255..781) v4 ==== 628+0+0 (unknown 1728308081 0 0) 0x7f0ab005e950 con 0x7f0ab810da20 2024-02-14T18:01:22.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:22.924+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=782}) v3 -- 0x7f0aa407c370 con 0x7f0ab810da20 2024-02-14T18:01:22.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:22.930+0000 7f0aaf7fe640 1 -- 172.21.15.178:0/595415330 <== mon.2 v1:172.21.15.178:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v781) v1 ==== 217+0+0 (unknown 3157439162 0 0) 0x7f0ab0055b10 con 0x7f0ab810da20 2024-02-14T18:01:22.932 INFO:tasks.workunit.client.0.smithi178.stderr:set-quota max_bytes = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b 2024-02-14T18:01:22.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:22.933+0000 7f0abcf3e640 1 -- 172.21.15.178:0/595415330 >> v1:172.21.15.178:6800/4205070280 conn(0x7f0aa406e9e0 legacy=0x7f0aa4070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:22.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:22.933+0000 7f0abcf3e640 1 -- 172.21.15.178:0/595415330 >> v1:172.21.15.178:6790/0 conn(0x7f0ab810da20 legacy=0x7f0ab81a44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:22.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:22.933+0000 7f0abcf3e640 1 -- 172.21.15.178:0/595415330 shutdown_connections 2024-02-14T18:01:22.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:22.933+0000 7f0abcf3e640 1 -- 172.21.15.178:0/595415330 >> 172.21.15.178:0/595415330 conn(0x7f0ab80fca70 msgr2=0x7f0ab810ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:01:22.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:22.933+0000 7f0abcf3e640 1 -- 172.21.15.178:0/595415330 shutdown_connections 2024-02-14T18:01:22.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:22.933+0000 7f0abcf3e640 1 -- 172.21.15.178:0/595415330 wait complete. 2024-02-14T18:01:22.944 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool set-quota c8dfcd28-16bd-46b5-a2ee-f0743e289c3b max_objects 0 2024-02-14T18:01:23.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.025+0000 7fea5d972640 1 -- 172.21.15.178:0/4212056073 >> v1:172.21.15.178:6790/0 conn(0x7fea5810c0f0 legacy=0x7fea5810e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:23.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.025+0000 7fea5d972640 1 -- 172.21.15.178:0/4212056073 shutdown_connections 2024-02-14T18:01:23.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.025+0000 7fea5d972640 1 -- 172.21.15.178:0/4212056073 >> 172.21.15.178:0/4212056073 conn(0x7fea58075210 msgr2=0x7fea58075620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:01:23.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.025+0000 7fea5d972640 1 -- 172.21.15.178:0/4212056073 shutdown_connections 2024-02-14T18:01:23.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.025+0000 7fea5d972640 1 -- 172.21.15.178:0/4212056073 wait complete. 2024-02-14T18:01:23.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.026+0000 7fea5d972640 1 Processor -- start 2024-02-14T18:01:23.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.026+0000 7fea5d972640 1 -- start start 2024-02-14T18:01:23.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.026+0000 7fea5d972640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fea581a3ce0 con 0x7fea58105680 2024-02-14T18:01:23.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.026+0000 7fea5d972640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fea581a4ea0 con 0x7fea58108520 2024-02-14T18:01:23.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.026+0000 7fea5d972640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fea581a60a0 con 0x7fea5810c0f0 2024-02-14T18:01:23.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.026+0000 7fea56ffd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fea58105680 0x7fea580fc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:35340/0 (socket says 172.21.15.178:35340) 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.026+0000 7fea56ffd640 1 -- 172.21.15.178:0/3884006706 learned_addr learned my addr 172.21.15.178:0/3884006706 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335735943 0 0) 0x7fea581a3ce0 con 0x7fea58105680 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea440056d0 con 0x7fea58105680 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309027068 0 0) 0x7fea581a60a0 con 0x7fea5810c0f0 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea581a3ce0 con 0x7fea5810c0f0 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820204164 0 0) 0x7fea581a4ea0 con 0x7fea58108520 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea581a60a0 con 0x7fea58108520 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2812007929 0 0) 0x7fea440056d0 con 0x7fea58105680 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fea581a4ea0 con 0x7fea58105680 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1797680585 0 0) 0x7fea581a3ce0 con 0x7fea5810c0f0 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fea440056d0 con 0x7fea5810c0f0 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fea400019f0 con 0x7fea58105680 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fea50001c60 con 0x7fea5810c0f0 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 199631201 0 0) 0x7fea581a60a0 con 0x7fea58108520 2024-02-14T18:01:23.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fea581a3ce0 con 0x7fea58108520 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fea48001830 con 0x7fea58108520 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3840246583 0 0) 0x7fea440056d0 con 0x7fea5810c0f0 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 >> v1:172.21.15.184:6789/0 conn(0x7fea58108520 legacy=0x7fea580fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 >> v1:172.21.15.178:6789/0 conn(0x7fea58105680 legacy=0x7fea580fc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea581a72a0 con 0x7fea5810c0f0 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5d972640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea581a6290 con 0x7fea5810c0f0 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5d972640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fea581a6720 con 0x7fea5810c0f0 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7fea50001480 con 0x7fea5810c0f0 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.027+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7fea50004ea0 con 0x7fea5810c0f0 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.028+0000 7fea5d972640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea38005080 con 0x7fea5810c0f0 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.029+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7fea50003970 con 0x7fea5810c0f0 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.029+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(781..781 src has 255..781) v4 ==== 8218+0+0 (unknown 1209100130 0 0) 0x7fea50088220 con 0x7fea5810c0f0 2024-02-14T18:01:23.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.029+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=782}) v3 -- 0x7fea440056d0 con 0x7fea5810c0f0 2024-02-14T18:01:23.035 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.033+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fea50054d30 con 0x7fea5810c0f0 2024-02-14T18:01:23.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.208+0000 7fea5d972640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fea38005330 con 0x7fea5810c0f0 2024-02-14T18:01:23.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:22 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:22 smithi184 ceph-mon[33536]: from='client.57254 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:01:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:22 smithi184 ceph-mon[33536]: osdmap e780: 8 total, 8 up, 8 in 2024-02-14T18:01:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:22 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/595415330' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:23.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:22 smithi184 ceph-mon[33536]: from='client.57254 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:23.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:22 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:23.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:22 smithi178 ceph-mon[39811]: from='client.57254 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:01:23.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:22 smithi178 ceph-mon[39811]: osdmap e780: 8 total, 8 up, 8 in 2024-02-14T18:01:23.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:22 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/595415330' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:23.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:22 smithi178 ceph-mon[39811]: from='client.57254 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:22 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:22 smithi178 ceph-mon[30321]: from='client.57254 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:01:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:22 smithi178 ceph-mon[30321]: osdmap e780: 8 total, 8 up, 8 in 2024-02-14T18:01:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:22 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/595415330' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:23.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:22 smithi178 ceph-mon[30321]: from='client.57254 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:01:23.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.930+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 10 ==== osd_map(782..782 src has 255..782) v4 ==== 628+0+0 (unknown 4220154395 0 0) 0x7fea50055660 con 0x7fea5810c0f0 2024-02-14T18:01:23.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.930+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=783}) v3 -- 0x7fea581a4ea0 con 0x7fea5810c0f0 2024-02-14T18:01:23.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:23.934+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v782) v1 ==== 221+0+0 (unknown 1809350265 0 0) 0x7fea500530a0 con 0x7fea5810c0f0 2024-02-14T18:01:24.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:24.044+0000 7fea5d972640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fea38000c20 con 0x7fea5810c0f0 2024-02-14T18:01:24.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:23 smithi184 ceph-mon[33536]: pgmap v1271: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:24.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:23 smithi184 ceph-mon[33536]: from='client.57254 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:01:24.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:23 smithi184 ceph-mon[33536]: osdmap e781: 8 total, 8 up, 8 in 2024-02-14T18:01:24.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:23 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3884006706' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:24.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:23 smithi184 ceph-mon[33536]: from='client.57260 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:24.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:23 smithi178 ceph-mon[39811]: pgmap v1271: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:24.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:23 smithi178 ceph-mon[39811]: from='client.57254 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:01:24.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:23 smithi178 ceph-mon[39811]: osdmap e781: 8 total, 8 up, 8 in 2024-02-14T18:01:24.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:23 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3884006706' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:24.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:23 smithi178 ceph-mon[39811]: from='client.57260 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:24.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:23 smithi178 ceph-mon[30321]: pgmap v1271: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:24.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:23 smithi178 ceph-mon[30321]: from='client.57254 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:01:24.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:23 smithi178 ceph-mon[30321]: osdmap e781: 8 total, 8 up, 8 in 2024-02-14T18:01:24.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:23 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3884006706' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:24.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:23 smithi178 ceph-mon[30321]: from='client.57260 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:24.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:24.935+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 12 ==== osd_map(783..783 src has 255..783) v4 ==== 628+0+0 (unknown 4192201524 0 0) 0x7fea500590b0 con 0x7fea5810c0f0 2024-02-14T18:01:24.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:24.935+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=784}) v3 -- 0x7fea581a60a0 con 0x7fea5810c0f0 2024-02-14T18:01:24.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:24.940+0000 7fea5c970640 1 -- 172.21.15.178:0/3884006706 <== mon.2 v1:172.21.15.178:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b v783) v1 ==== 221+0+0 (unknown 3951603608 0 0) 0x7fea500548c0 con 0x7fea5810c0f0 2024-02-14T18:01:24.942 INFO:tasks.workunit.client.0.smithi178.stderr:set-quota max_objects = 0 for pool c8dfcd28-16bd-46b5-a2ee-f0743e289c3b 2024-02-14T18:01:24.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:24.944+0000 7fea5d972640 1 -- 172.21.15.178:0/3884006706 >> v1:172.21.15.178:6800/4205070280 conn(0x7fea44070be0 legacy=0x7fea440730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:24.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:24.944+0000 7fea5d972640 1 -- 172.21.15.178:0/3884006706 >> v1:172.21.15.178:6790/0 conn(0x7fea5810c0f0 legacy=0x7fea581a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:24.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:24.944+0000 7fea5d972640 1 -- 172.21.15.178:0/3884006706 shutdown_connections 2024-02-14T18:01:24.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:24.944+0000 7fea5d972640 1 -- 172.21.15.178:0/3884006706 >> 172.21.15.178:0/3884006706 conn(0x7fea58075210 msgr2=0x7fea581064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:01:24.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:24.944+0000 7fea5d972640 1 -- 172.21.15.178:0/3884006706 shutdown_connections 2024-02-14T18:01:24.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:24.944+0000 7fea5d972640 1 -- 172.21.15.178:0/3884006706 wait complete. 2024-02-14T18:01:24.956 INFO:tasks.workunit.client.0.smithi178.stderr:+ sleep 30 2024-02-14T18:01:25.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:24 smithi184 ceph-mon[33536]: from='client.57260 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:01:25.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:24 smithi184 ceph-mon[33536]: osdmap e782: 8 total, 8 up, 8 in 2024-02-14T18:01:25.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:24 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3884006706' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:25.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:24 smithi184 ceph-mon[33536]: from='client.57260 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:24 smithi178 ceph-mon[39811]: from='client.57260 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:01:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:24 smithi178 ceph-mon[39811]: osdmap e782: 8 total, 8 up, 8 in 2024-02-14T18:01:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:24 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3884006706' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:25.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:24 smithi178 ceph-mon[39811]: from='client.57260 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:25.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:24 smithi178 ceph-mon[30321]: from='client.57260 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:01:25.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:24 smithi178 ceph-mon[30321]: osdmap e782: 8 total, 8 up, 8 in 2024-02-14T18:01:25.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:24 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3884006706' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:25.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:24 smithi178 ceph-mon[30321]: from='client.57260 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:01:25.938 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:25 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:01:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:25.939 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:25 smithi178 ceph-mon[39811]: pgmap v1274: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:25.939 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:25 smithi178 ceph-mon[39811]: from='client.57260 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:01:25.939 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:25 smithi178 ceph-mon[30321]: pgmap v1274: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:25.939 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:25 smithi178 ceph-mon[30321]: from='client.57260 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:01:25.939 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:25 smithi178 ceph-mon[30321]: osdmap e783: 8 total, 8 up, 8 in 2024-02-14T18:01:26.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:25 smithi184 ceph-mon[33536]: pgmap v1274: 188 pgs: 188 active+clean; 858 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:26.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:25 smithi184 ceph-mon[33536]: from='client.57260 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:01:26.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:25 smithi184 ceph-mon[33536]: osdmap e783: 8 total, 8 up, 8 in 2024-02-14T18:01:26.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:25 smithi178 ceph-mon[39811]: osdmap e783: 8 total, 8 up, 8 in 2024-02-14T18:01:28.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:27 smithi184 ceph-mon[33536]: pgmap v1276: 188 pgs: 188 active+clean; 870 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-02-14T18:01:28.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:27 smithi178 ceph-mon[39811]: pgmap v1276: 188 pgs: 188 active+clean; 870 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-02-14T18:01:28.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:27 smithi178 ceph-mon[30321]: pgmap v1276: 188 pgs: 188 active+clean; 870 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-02-14T18:01:29.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:28 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:29.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:28 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:29.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:28 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:30.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:29 smithi184 ceph-mon[33536]: pgmap v1277: 188 pgs: 188 active+clean; 870 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-14T18:01:30.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:29 smithi178 ceph-mon[39811]: pgmap v1277: 188 pgs: 188 active+clean; 870 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-14T18:01:30.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:29 smithi178 ceph-mon[30321]: pgmap v1277: 188 pgs: 188 active+clean; 870 KiB data, 827 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-14T18:01:32.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:31 smithi184 ceph-mon[33536]: pgmap v1278: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-02-14T18:01:32.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:01:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:01:32.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:31 smithi178 ceph-mon[39811]: pgmap v1278: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-02-14T18:01:32.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:31 smithi178 ceph-mon[30321]: pgmap v1278: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-02-14T18:01:33.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:32 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:33.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:32 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:33.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:32 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:34.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:33 smithi184 ceph-mon[33536]: pgmap v1279: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-14T18:01:34.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:33 smithi178 ceph-mon[30321]: pgmap v1279: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-14T18:01:34.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:33 smithi178 ceph-mon[39811]: pgmap v1279: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-14T18:01:35.958 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:35 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:01:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:36.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:35 smithi184 ceph-mon[33536]: pgmap v1280: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-14T18:01:36.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:35 smithi178 ceph-mon[30321]: pgmap v1280: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-14T18:01:36.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:35 smithi178 ceph-mon[39811]: pgmap v1280: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-14T18:01:38.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:37 smithi184 ceph-mon[33536]: pgmap v1281: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-14T18:01:38.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:37 smithi178 ceph-mon[30321]: pgmap v1281: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-14T18:01:38.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:37 smithi178 ceph-mon[39811]: pgmap v1281: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-14T18:01:40.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:39 smithi184 ceph-mon[33536]: pgmap v1282: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:40.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:39 smithi178 ceph-mon[30321]: pgmap v1282: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:40.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:39 smithi178 ceph-mon[39811]: pgmap v1282: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:42.219 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:41 smithi184 ceph-mon[33536]: pgmap v1283: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:42.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:01:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:01:42.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:41 smithi178 ceph-mon[30321]: pgmap v1283: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:42.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:41 smithi178 ceph-mon[39811]: pgmap v1283: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:43.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:42 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:43.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:42 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:43.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:42 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:44.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:43 smithi178 ceph-mon[30321]: pgmap v1284: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:44.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:43 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:44.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:43 smithi178 ceph-mon[39811]: pgmap v1284: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:44.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:43 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:44.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:43 smithi184 ceph-mon[33536]: pgmap v1284: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:44.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:43 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:45.975 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:45 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:01:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:46.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:45 smithi178 ceph-mon[39811]: pgmap v1285: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:46.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:45 smithi178 ceph-mon[30321]: pgmap v1285: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:46.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:45 smithi184 ceph-mon[33536]: pgmap v1285: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:47.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:46 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:01:47.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:46 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:01:47.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:46 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:01:48.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:47 smithi178 ceph-mon[30321]: pgmap v1286: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:48.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:47 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:01:48.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:47 smithi178 ceph-mon[30321]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:01:48.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:47 smithi178 ceph-mon[30321]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:01:48.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:47 smithi178 ceph-mon[39811]: pgmap v1286: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:48.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:47 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:01:48.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:47 smithi178 ceph-mon[39811]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:01:48.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:47 smithi178 ceph-mon[39811]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:01:48.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:47 smithi184 ceph-mon[33536]: pgmap v1286: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:48.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:47 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:01:48.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:47 smithi184 ceph-mon[33536]: from='mgr.14595 v1:172.21.15.178:0/2818024841' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:01:48.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:47 smithi184 ceph-mon[33536]: from='mgr.14595 ' entity='mgr.y' 2024-02-14T18:01:50.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:49 smithi178 ceph-mon[30321]: pgmap v1287: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:50.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:49 smithi178 ceph-mon[39811]: pgmap v1287: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:50.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:49 smithi184 ceph-mon[33536]: pgmap v1287: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:52.218 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:51 smithi184 ceph-mon[33536]: pgmap v1288: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:52.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:01:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:01:52.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:51 smithi178 ceph-mon[30321]: pgmap v1288: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:52.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:51 smithi178 ceph-mon[39811]: pgmap v1288: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:53.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:52 smithi178 ceph-mon[30321]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:53.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:52 smithi178 ceph-mon[39811]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:53.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:52 smithi184 ceph-mon[33536]: from='client.24475 v1:172.21.15.184:0/1359372283' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:54.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:53 smithi178 ceph-mon[30321]: pgmap v1289: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:54.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:53 smithi178 ceph-mon[39811]: pgmap v1289: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:54.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:53 smithi184 ceph-mon[33536]: pgmap v1289: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:01:54.957 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool delete c8dfcd28-16bd-46b5-a2ee-f0743e289c3b c8dfcd28-16bd-46b5-a2ee-f0743e289c3b --yes-i-really-really-mean-it 2024-02-14T18:01:55.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.050+0000 7f9099c9d640 1 -- 172.21.15.178:0/466781054 >> v1:172.21.15.178:6790/0 conn(0x7f9094100140 legacy=0x7f9094100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:55.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.050+0000 7f9099c9d640 1 -- 172.21.15.178:0/466781054 shutdown_connections 2024-02-14T18:01:55.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.050+0000 7f9099c9d640 1 -- 172.21.15.178:0/466781054 >> 172.21.15.178:0/466781054 conn(0x7f90940fb980 msgr2=0x7f90940fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:01:55.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.050+0000 7f9099c9d640 1 -- 172.21.15.178:0/466781054 shutdown_connections 2024-02-14T18:01:55.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.050+0000 7f9099c9d640 1 -- 172.21.15.178:0/466781054 wait complete. 2024-02-14T18:01:55.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9099c9d640 1 Processor -- start 2024-02-14T18:01:55.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9099c9d640 1 -- start start 2024-02-14T18:01:55.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9099c9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f90941a5d80 con 0x7f909410c930 2024-02-14T18:01:55.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9099c9d640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f90941a6f80 con 0x7f9094108dc0 2024-02-14T18:01:55.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9099c9d640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f90941a8180 con 0x7f9094100140 2024-02-14T18:01:55.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f90937fe640 1 --1- >> v1:172.21.15.178:6790/0 conn(0x7f9094100140 0x7f90941052c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6790/0 says I am v1:172.21.15.178:43328/0 (socket says 172.21.15.178:43328) 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f90937fe640 1 -- 172.21.15.178:0/3629132464 learned_addr learned my addr 172.21.15.178:0/3629132464 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9093fff640 1 --1- 172.21.15.178:0/3629132464 >> v1:172.21.15.178:6789/0 conn(0x7f909410c930 0x7f90941a4480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:53442/0 (socket says 172.21.15.178:53442) 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239600237 0 0) 0x7f90941a5d80 con 0x7f909410c930 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f907c003520 con 0x7f909410c930 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011117344 0 0) 0x7f90941a8180 con 0x7f9094100140 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90941a5d80 con 0x7f9094100140 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320814033 0 0) 0x7f90941a6f80 con 0x7f9094108dc0 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.051+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90941a8180 con 0x7f9094108dc0 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2143091029 0 0) 0x7f907c003520 con 0x7f909410c930 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f90941a6f80 con 0x7f909410c930 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f908c001c10 con 0x7f909410c930 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1237133008 0 0) 0x7f90941a5d80 con 0x7f9094100140 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f907c003520 con 0x7f9094100140 2024-02-14T18:01:55.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f90840043f0 con 0x7f9094100140 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1529980377 0 0) 0x7f90941a8180 con 0x7f9094108dc0 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f90941a5d80 con 0x7f9094108dc0 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f9088001970 con 0x7f9094108dc0 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1485751913 0 0) 0x7f90941a6f80 con 0x7f909410c930 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 >> v1:172.21.15.178:6790/0 conn(0x7f9094100140 legacy=0x7f90941052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 >> v1:172.21.15.184:6789/0 conn(0x7f9094108dc0 legacy=0x7f90941a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90941a9380 con 0x7f909410c930 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9099c9d640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90941a8370 con 0x7f909410c930 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9099c9d640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f90941a8800 con 0x7f909410c930 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f908c003e20 con 0x7f909410c930 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.052+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f908c005e90 con 0x7f909410c930 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.053+0000 7f9099c9d640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9080005080 con 0x7f909410c930 2024-02-14T18:01:55.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.053+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f908c006100 con 0x7f909410c930 2024-02-14T18:01:55.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.054+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 8 ==== osd_map(783..783 src has 255..783) v4 ==== 8218+0+0 (unknown 3058175063 0 0) 0x7f908c089330 con 0x7f909410c930 2024-02-14T18:01:55.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.054+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=784}) v3 -- 0x7f90941a6f80 con 0x7f909410c930 2024-02-14T18:01:55.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.059+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f908c055e40 con 0x7f909410c930 2024-02-14T18:01:55.236 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.235+0000 7f9099c9d640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9080005330 con 0x7f909410c930 2024-02-14T18:01:55.996 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:55 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: ::ffff:172.21.15.184 - - [14/Feb/2024:18:01:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:56.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:55.999+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true}]=0 pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' removed v784) v1 ==== 248+0+0 (unknown 3550884292 0 0) 0x7f908c0541b0 con 0x7f909410c930 2024-02-14T18:01:56.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.000+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 11 ==== osd_map(784..784 src has 255..784) v4 ==== 296+0+0 (unknown 4044109428 0 0) 0x7f908c056770 con 0x7f909410c930 2024-02-14T18:01:56.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.000+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6789/0 -- mon_subscribe({osdmap=785}) v3 -- 0x7f907c003520 con 0x7f909410c930 2024-02-14T18:01:56.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.119+0000 7f9099c9d640 1 -- 172.21.15.178:0/3629132464 --> v1:172.21.15.178:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9080005d60 con 0x7f909410c930 2024-02-14T18:01:56.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.120+0000 7f9090ff9640 1 -- 172.21.15.178:0/3629132464 <== mon.0 v1:172.21.15.178:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true}]=0 pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' does not exist v784) v1 ==== 255+0+0 (unknown 371856778 0 0) 0x7f908c0559d0 con 0x7f909410c930 2024-02-14T18:01:56.121 INFO:tasks.workunit.client.0.smithi178.stderr:pool 'c8dfcd28-16bd-46b5-a2ee-f0743e289c3b' does not exist 2024-02-14T18:01:56.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.122+0000 7f9099c9d640 1 -- 172.21.15.178:0/3629132464 >> v1:172.21.15.178:6800/4205070280 conn(0x7f907c06e9d0 legacy=0x7f907c070e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:56.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.122+0000 7f9099c9d640 1 -- 172.21.15.178:0/3629132464 >> v1:172.21.15.178:6789/0 conn(0x7f909410c930 legacy=0x7f90941a4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:56.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.122+0000 7f9099c9d640 1 -- 172.21.15.178:0/3629132464 shutdown_connections 2024-02-14T18:01:56.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.122+0000 7f9099c9d640 1 -- 172.21.15.178:0/3629132464 >> 172.21.15.178:0/3629132464 conn(0x7f90940fb980 msgr2=0x7f909410fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:01:56.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.122+0000 7f9099c9d640 1 -- 172.21.15.178:0/3629132464 shutdown_connections 2024-02-14T18:01:56.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.122+0000 7f9099c9d640 1 -- 172.21.15.178:0/3629132464 wait complete. 2024-02-14T18:01:56.132 INFO:tasks.workunit.client.0.smithi178.stderr:+ ceph osd pool delete 9abf3c01-ef88-436b-b12e-c5decaf64871 9abf3c01-ef88-436b-b12e-c5decaf64871 --yes-i-really-really-mean-it 2024-02-14T18:01:56.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.213+0000 7f9f507ac640 1 -- 172.21.15.178:0/11113694 >> v1:172.21.15.178:6789/0 conn(0x7f9f48108dc0 legacy=0x7f9f4810b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:56.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.213+0000 7f9f507ac640 1 -- 172.21.15.178:0/11113694 shutdown_connections 2024-02-14T18:01:56.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.213+0000 7f9f507ac640 1 -- 172.21.15.178:0/11113694 >> 172.21.15.178:0/11113694 conn(0x7f9f480fb980 msgr2=0x7f9f480fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:01:56.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.214+0000 7f9f507ac640 1 -- 172.21.15.178:0/11113694 shutdown_connections 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.214+0000 7f9f507ac640 1 -- 172.21.15.178:0/11113694 wait complete. 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.214+0000 7f9f507ac640 1 Processor -- start 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.214+0000 7f9f507ac640 1 -- start start 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.214+0000 7f9f507ac640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9f481a5ea0 con 0x7f9f48100140 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.214+0000 7f9f507ac640 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9f481a70a0 con 0x7f9f48108dc0 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.214+0000 7f9f507ac640 1 -- --> v1:172.21.15.178:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9f481a82a0 con 0x7f9f4810c930 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.214+0000 7f9f4e548640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f9f48100140 0x7f9f481053f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:53452/0 (socket says 172.21.15.178:53452) 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.214+0000 7f9f4e548640 1 -- 172.21.15.178:0/1197480697 learned_addr learned my addr 172.21.15.178:0/1197480697 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124714229 0 0) 0x7f9f481a5ea0 con 0x7f9f48100140 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f34003520 con 0x7f9f48100140 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590724739 0 0) 0x7f9f481a82a0 con 0x7f9f4810c930 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f481a5ea0 con 0x7f9f4810c930 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355561200 0 0) 0x7f9f481a70a0 con 0x7f9f48108dc0 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f481a82a0 con 0x7f9f48108dc0 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 777529552 0 0) 0x7f9f34003520 con 0x7f9f48100140 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f481a70a0 con 0x7f9f48100140 2024-02-14T18:01:56.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 686368119 0 0) 0x7f9f481a5ea0 con 0x7f9f4810c930 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f34003520 con 0x7f9f4810c930 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f9f44001930 con 0x7f9f48100140 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f9f40001bb0 con 0x7f9f4810c930 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2836632423 0 0) 0x7f9f481a82a0 con 0x7f9f48108dc0 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.184:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f481a5ea0 con 0x7f9f48108dc0 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f9f38004370 con 0x7f9f48108dc0 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3393638777 0 0) 0x7f9f34003520 con 0x7f9f4810c930 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 >> v1:172.21.15.184:6789/0 conn(0x7f9f48108dc0 legacy=0x7f9f481a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 >> v1:172.21.15.178:6789/0 conn(0x7f9f48100140 legacy=0x7f9f481053f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f481a94a0 con 0x7f9f4810c930 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f507ac640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f481a8490 con 0x7f9f4810c930 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f507ac640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f481a8920 con 0x7f9f4810c930 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 2884372430 0 0) 0x7f9f40003e00 con 0x7f9f4810c930 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.215+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2295542603 0 0) 0x7f9f40004b50 con 0x7f9f4810c930 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.216+0000 7f9f507ac640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2c005080 con 0x7f9f4810c930 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.216+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 7 ==== mgrmap(e 21) v1 ==== 91430+0+0 (unknown 488189149 0 0) 0x7f9f40003970 con 0x7f9f4810c930 2024-02-14T18:01:56.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.217+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 8 ==== osd_map(784..784 src has 255..784) v4 ==== 7830+0+0 (unknown 2063362815 0 0) 0x7f9f40087f30 con 0x7f9f4810c930 2024-02-14T18:01:56.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.217+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=785}) v3 -- 0x7f9f34003520 con 0x7f9f4810c930 2024-02-14T18:01:56.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.221+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f9f40054c10 con 0x7f9f4810c930 2024-02-14T18:01:56.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:55 smithi178 ceph-mon[39811]: pgmap v1290: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:56.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:55 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3629132464' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:56.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:55 smithi178 ceph-mon[30321]: pgmap v1290: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:56.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:55 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3629132464' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:56.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:56.396+0000 7f9f507ac640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9f2c005330 con 0x7f9f4810c930 2024-02-14T18:01:56.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:55 smithi184 ceph-mon[33536]: pgmap v1290: 188 pgs: 188 active+clean; 870 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:56.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:55 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3629132464' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:57.008 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:57.006+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 10 ==== osd_map(785..785 src has 255..785) v4 ==== 296+0+0 (unknown 179617761 0 0) 0x7f9f40055540 con 0x7f9f4810c930 2024-02-14T18:01:57.008 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:57.006+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6790/0 -- mon_subscribe({osdmap=786}) v3 -- 0x7f9f481a70a0 con 0x7f9f4810c930 2024-02-14T18:01:57.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:57.012+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]=0 pool '9abf3c01-ef88-436b-b12e-c5decaf64871' removed v785) v1 ==== 248+0+0 (unknown 3565574332 0 0) 0x7f9f40052f80 con 0x7f9f4810c930 2024-02-14T18:01:57.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:57.128+0000 7f9f507ac640 1 -- 172.21.15.178:0/1197480697 --> v1:172.21.15.178:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9f2c001140 con 0x7f9f4810c930 2024-02-14T18:01:57.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:57.129+0000 7f9f3f7fe640 1 -- 172.21.15.178:0/1197480697 <== mon.2 v1:172.21.15.178:6790/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]=0 pool '9abf3c01-ef88-436b-b12e-c5decaf64871' does not exist v785) v1 ==== 255+0+0 (unknown 1361286502 0 0) 0x7f9f400547a0 con 0x7f9f4810c930 2024-02-14T18:01:57.130 INFO:tasks.workunit.client.0.smithi178.stderr:pool '9abf3c01-ef88-436b-b12e-c5decaf64871' does not exist 2024-02-14T18:01:57.132 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:57.131+0000 7f9f507ac640 1 -- 172.21.15.178:0/1197480697 >> v1:172.21.15.178:6800/4205070280 conn(0x7f9f3406ea20 legacy=0x7f9f34070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:57.132 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:57.131+0000 7f9f507ac640 1 -- 172.21.15.178:0/1197480697 >> v1:172.21.15.178:6790/0 conn(0x7f9f4810c930 legacy=0x7f9f481a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:01:57.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:57.131+0000 7f9f507ac640 1 -- 172.21.15.178:0/1197480697 shutdown_connections 2024-02-14T18:01:57.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:57.131+0000 7f9f507ac640 1 -- 172.21.15.178:0/1197480697 >> 172.21.15.178:0/1197480697 conn(0x7f9f480fb980 msgr2=0x7f9f480695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:01:57.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:57.131+0000 7f9f507ac640 1 -- 172.21.15.178:0/1197480697 shutdown_connections 2024-02-14T18:01:57.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-02-14T18:01:57.131+0000 7f9f507ac640 1 -- 172.21.15.178:0/1197480697 wait complete. 2024-02-14T18:01:57.141 INFO:tasks.workunit.client.0.smithi178.stdout:OK 2024-02-14T18:01:57.142 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo OK 2024-02-14T18:01:57.143 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-14T18:01:57.143 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-14T18:01:57.213 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-02-14T18:01:57.213 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-14T18:01:57.277 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3629132464' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:01:57.277 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:57 smithi178 ceph-mon[30321]: osdmap e784: 8 total, 8 up, 8 in 2024-02-14T18:01:57.277 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/3629132464' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:57.277 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:57 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1197480697' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:57.277 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:57 smithi178 ceph-mon[30321]: from='client.57272 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:57.277 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3629132464' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:01:57.277 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:57 smithi178 ceph-mon[39811]: osdmap e784: 8 total, 8 up, 8 in 2024-02-14T18:01:57.278 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/3629132464' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:57.278 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:57 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1197480697' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:57.278 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:57 smithi178 ceph-mon[39811]: from='client.57272 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:57.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3629132464' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:01:57.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:57 smithi184 ceph-mon[33536]: osdmap e784: 8 total, 8 up, 8 in 2024-02-14T18:01:57.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/3629132464' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "pool2": "c8dfcd28-16bd-46b5-a2ee-f0743e289c3b", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:57.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:57 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1197480697' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:57.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:57 smithi184 ceph-mon[33536]: from='client.57272 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:57.523 DEBUG:teuthology.parallel:result is None 2024-02-14T18:01:57.523 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-14T18:01:57.590 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-14T18:01:57.591 DEBUG:teuthology.orchestra.run.smithi178:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-14T18:01:57.656 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-14T18:01:57.656 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-14T18:01:57.670 INFO:tasks.cephadm:Teardown begin 2024-02-14T18:01:57.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T18:01:57.723 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T18:01:57.758 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-14T18:01:57.758 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-14T18:01:57.777 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-14T18:01:57.815 INFO:tasks.cephadm:Stopping all daemons... 2024-02-14T18:01:57.815 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-14T18:01:57.815 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.a 2024-02-14T18:01:58.008 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:58 smithi184 ceph-mon[33536]: pgmap v1292: 176 pgs: 176 active+clean; 697 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:58.009 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:58 smithi184 ceph-mon[33536]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:01:58.142 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[39811]: pgmap v1292: 176 pgs: 176 active+clean; 697 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:58.143 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[39811]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:01:58.143 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[39811]: from='client.57272 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:01:58.143 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[39811]: osdmap e785: 8 total, 8 up, 8 in 2024-02-14T18:01:58.143 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[39811]: from='client.? v1:172.21.15.178:0/1197480697' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:58.143 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[39811]: from='client.57272 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:58.143 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:57 smithi178 systemd[1]: Stopping Ceph mon.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:01:58.143 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[30321]: pgmap v1292: 176 pgs: 176 active+clean; 697 KiB data, 828 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:58.143 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[30321]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:01:58.143 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[30321]: from='client.57272 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:01:58.143 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[30321]: osdmap e785: 8 total, 8 up, 8 in 2024-02-14T18:01:58.143 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[30321]: from='client.? v1:172.21.15.178:0/1197480697' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:58.144 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-mon[30321]: from='client.57272 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:58.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:58 smithi184 ceph-mon[33536]: from='client.57272 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:01:58.468 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:58 smithi184 ceph-mon[33536]: osdmap e785: 8 total, 8 up, 8 in 2024-02-14T18:01:58.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:58 smithi184 ceph-mon[33536]: from='client.? v1:172.21.15.178:0/1197480697' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:58.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:01:58 smithi184 ceph-mon[33536]: from='client.57272 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "9abf3c01-ef88-436b-b12e-c5decaf64871", "pool2": "9abf3c01-ef88-436b-b12e-c5decaf64871", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:01:58.498 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a[30296]: 2024-02-14T18:01:58.140+0000 7ff7c7d39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:01:58.499 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:58 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a[30296]: 2024-02-14T18:01:58.140+0000 7ff7c7d39700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-14T18:01:58.499 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:58 smithi178 podman[152780]: 2024-02-14 18:01:58.22340724 +0000 UTC m=+0.225140983 container died 46ad1d3e6f4404dd1ea1a53a400fcb84fa8160fbe44b330b5216e28512550b2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-14T18:01:58.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:58 smithi178 podman[152780]: 2024-02-14 18:01:58.680478964 +0000 UTC m=+0.682212688 container cleanup 46ad1d3e6f4404dd1ea1a53a400fcb84fa8160fbe44b330b5216e28512550b2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T18:01:58.832 INFO:journalctl@ceph.mon.a.smithi178.stdout:Feb 14 18:01:58 smithi178 bash[152780]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-a 2024-02-14T18:01:59.148 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.a.service' 2024-02-14T18:01:59.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:01:59.193 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-14T18:01:59.193 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-14T18:01:59.193 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.c 2024-02-14T18:01:59.539 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:59 smithi178 systemd[1]: Stopping Ceph mon.c for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:01:59.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-c[39788]: 2024-02-14T18:01:59.537+0000 7fba120f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:01:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-c[39788]: 2024-02-14T18:01:59.537+0000 7fba120f0700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-14T18:01:59.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:59 smithi178 podman[152911]: 2024-02-14 18:01:59.569349799 +0000 UTC m=+0.158237476 container died ede2979ba29a19db2de016e067a6ce06720446cea0a8ef6de9cd463f09155394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-14T18:01:59.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:18:01:59] ENGINE Bus STOPPING 2024-02-14T18:01:59.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:18:01:59] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T18:01:59.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:18:01:59] ENGINE Bus STOPPED 2024-02-14T18:01:59.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:18:01:59] ENGINE Bus STARTING 2024-02-14T18:01:59.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:18:01:59] ENGINE Serving on http://:::9283 2024-02-14T18:01:59.833 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:01:59 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: [14/Feb/2024:18:01:59] ENGINE Bus STARTED 2024-02-14T18:02:00.212 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:59 smithi178 podman[152911]: 2024-02-14 18:01:59.95222349 +0000 UTC m=+0.541111167 container cleanup ede2979ba29a19db2de016e067a6ce06720446cea0a8ef6de9cd463f09155394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-14T18:02:00.213 INFO:journalctl@ceph.mon.c.smithi178.stdout:Feb 14 18:01:59 smithi178 bash[152911]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-c 2024-02-14T18:02:00.364 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.c.service' 2024-02-14T18:02:00.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:02:00.400 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-14T18:02:00.401 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-14T18:02:00.401 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.b 2024-02-14T18:02:00.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:02:00 smithi184 systemd[1]: Stopping Ceph mon.b for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:02:00.968 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:02:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-b[33514]: 2024-02-14T18:02:00.716+0000 7f11bff86700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:02:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:02:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-b[33514]: 2024-02-14T18:02:00.716+0000 7f11bff86700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-14T18:02:00.969 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:02:00 smithi184 podman[72644]: 2024-02-14 18:02:00.891796999 +0000 UTC m=+0.294376525 container died 47a8f037abf289a0285c566e75c285c6423d09afebaf2ac946eea5e1d52f429e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-14T18:02:01.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:02:01 smithi184 podman[72644]: 2024-02-14 18:02:01.298610078 +0000 UTC m=+0.701189611 container cleanup 47a8f037abf289a0285c566e75c285c6423d09afebaf2ac946eea5e1d52f429e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-14T18:02:01.469 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 14 18:02:01 smithi184 bash[72644]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mon-b 2024-02-14T18:02:01.640 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mon.b.service' 2024-02-14T18:02:01.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:02:01.686 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-14T18:02:01.686 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-14T18:02:01.687 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mgr.y 2024-02-14T18:02:01.994 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:02:01 smithi178 systemd[1]: Stopping Ceph mgr.y for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:02:02.218 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:02:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug there is no tcmu-runner data available 2024-02-14T18:02:02.332 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:02:01 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T18:02:01.991+0000 7f2dd269c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:02:02.332 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:02:01 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y[30579]: 2024-02-14T18:02:01.991+0000 7f2dd269c700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-14T18:02:02.333 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:02:02 smithi178 podman[153055]: 2024-02-14 18:02:02.025144261 +0000 UTC m=+0.146646274 container died 8e89fc86ca955baa915fb52ae9c8412eb23942aee79c419a6ca33369ba717773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-14T18:02:02.675 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:02:02 smithi178 podman[153055]: 2024-02-14 18:02:02.440046989 +0000 UTC m=+0.561549009 container cleanup 8e89fc86ca955baa915fb52ae9c8412eb23942aee79c419a6ca33369ba717773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-14T18:02:02.675 INFO:journalctl@ceph.mgr.y.smithi178.stdout:Feb 14 18:02:02 smithi178 bash[153055]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-y 2024-02-14T18:02:02.801 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mgr.y.service' 2024-02-14T18:02:02.837 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:02:02.838 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-14T18:02:02.838 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-14T18:02:02.838 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mgr.x 2024-02-14T18:02:03.218 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 18:02:02 smithi184 systemd[1]: Stopping Ceph mgr.x for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:02:03.623 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 18:02:03 smithi184 podman[72776]: 2024-02-14 18:02:03.353820858 +0000 UTC m=+0.323070031 container died 8a687eaae7d637e516d9bdab7fc7089d859adb935ab00ea5480a9bc3ba512998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-02-14T18:02:03.889 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 18:02:03 smithi184 podman[72776]: 2024-02-14 18:02:03.688246389 +0000 UTC m=+0.657495562 container cleanup 8a687eaae7d637e516d9bdab7fc7089d859adb935ab00ea5480a9bc3ba512998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-14T18:02:03.890 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 14 18:02:03 smithi184 bash[72776]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-mgr-x 2024-02-14T18:02:04.110 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@mgr.x.service' 2024-02-14T18:02:04.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:02:04.149 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-14T18:02:04.149 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-14T18:02:04.149 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.0 2024-02-14T18:02:04.514 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:04 smithi178 systemd[1]: Stopping Ceph osd.0 for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:02:04.832 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:04 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0[44215]: 2024-02-14T18:02:04.513+0000 7efd4b481700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:02:04.832 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:04 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0[44215]: 2024-02-14T18:02:04.513+0000 7efd4b481700 -1 osd.0 785 *** Got signal Terminated *** 2024-02-14T18:02:04.833 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:04 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0[44215]: 2024-02-14T18:02:04.513+0000 7efd4b481700 -1 osd.0 785 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:02:10.330 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:10 smithi178 podman[153189]: 2024-02-14 18:02:10.032551752 +0000 UTC m=+5.691805401 container died 0406a441d77a47a279d6a3db9ed5500cde036e29035b64b7e6702cd37b3154ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-14T18:02:10.582 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:10 smithi178 podman[153189]: 2024-02-14 18:02:10.411101712 +0000 UTC m=+6.070355358 container cleanup 0406a441d77a47a279d6a3db9ed5500cde036e29035b64b7e6702cd37b3154ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-14T18:02:10.582 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:10 smithi178 bash[153189]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0 2024-02-14T18:02:11.013 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:10 smithi178 podman[153209]: 2024-02-14 18:02:10.595646212 +0000 UTC m=+0.557886757 container remove 0406a441d77a47a279d6a3db9ed5500cde036e29035b64b7e6702cd37b3154ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-14T18:02:11.333 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:11 smithi178 podman[153307]: 2024-02-14 18:02:11.012968876 +0000 UTC m=+0.106368603 container create 0a77e302a25e6d985bf776450332ce21ad83e63ac7e8398f84bd817e07de0067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-14T18:02:11.333 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:11 smithi178 podman[153307]: 2024-02-14 18:02:10.924628518 +0000 UTC m=+0.018028253 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:02:11.333 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:11 smithi178 podman[153307]: 2024-02-14 18:02:11.249204424 +0000 UTC m=+0.342604166 container init 0a77e302a25e6d985bf776450332ce21ad83e63ac7e8398f84bd817e07de0067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD) 2024-02-14T18:02:11.333 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:11 smithi178 podman[153307]: 2024-02-14 18:02:11.25525476 +0000 UTC m=+0.348654487 container start 0a77e302a25e6d985bf776450332ce21ad83e63ac7e8398f84bd817e07de0067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-02-14T18:02:11.832 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:11 smithi178 podman[153307]: 2024-02-14 18:02:11.373582656 +0000 UTC m=+0.466982394 container attach 0a77e302a25e6d985bf776450332ce21ad83e63ac7e8398f84bd817e07de0067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-14T18:02:11.833 INFO:journalctl@ceph.osd.0.smithi178.stdout:Feb 14 18:02:11 smithi178 podman[153307]: 2024-02-14 18:02:11.543993222 +0000 UTC m=+0.637392949 container died 0a77e302a25e6d985bf776450332ce21ad83e63ac7e8398f84bd817e07de0067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True) 2024-02-14T18:02:12.137 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.0.service' 2024-02-14T18:02:12.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:02:12.180 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-14T18:02:12.180 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-14T18:02:12.181 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.1 2024-02-14T18:02:12.518 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:12 smithi178 systemd[1]: Stopping Ceph osd.1 for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:02:12.832 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:12 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1[50608]: 2024-02-14T18:02:12.516+0000 7f33e04cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:02:12.832 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:12 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1[50608]: 2024-02-14T18:02:12.516+0000 7f33e04cc700 -1 osd.1 785 *** Got signal Terminated *** 2024-02-14T18:02:12.832 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:12 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1[50608]: 2024-02-14T18:02:12.516+0000 7f33e04cc700 -1 osd.1 785 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:02:13.219 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:02:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:02:12.942Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.178:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.178:8765: connect: connection refused" 2024-02-14T18:02:13.219 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:02:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:02:12.943Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.178:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.178:8765: connect: connection refused" 2024-02-14T18:02:13.219 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:02:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:02:12.943Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.178:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.178:8765: connect: connection refused" 2024-02-14T18:02:13.219 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:02:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:02:12.944Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.178:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.178:8765: connect: connection refused" 2024-02-14T18:02:13.832 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:13.571Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused" 2024-02-14T18:02:13.832 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:13.571Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-14T18:02:18.257 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:17 smithi178 podman[153441]: 2024-02-14 18:02:17.976119748 +0000 UTC m=+5.573996839 container died 8de96f2d36209e588a4d0cff7159343c48b088caa70a4ec694df166d2d7ad7d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-14T18:02:18.583 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:18 smithi178 podman[153441]: 2024-02-14 18:02:18.354658743 +0000 UTC m=+5.952535834 container cleanup 8de96f2d36209e588a4d0cff7159343c48b088caa70a4ec694df166d2d7ad7d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-14T18:02:18.583 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:18 smithi178 bash[153441]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1 2024-02-14T18:02:18.974 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:18 smithi178 podman[153463]: 2024-02-14 18:02:18.610428357 +0000 UTC m=+0.637815590 container remove 8de96f2d36209e588a4d0cff7159343c48b088caa70a4ec694df166d2d7ad7d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1, ceph=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-14T18:02:19.227 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:18 smithi178 podman[153560]: 2024-02-14 18:02:18.877003303 +0000 UTC m=+0.018529130 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:02:19.227 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:19 smithi178 podman[153560]: 2024-02-14 18:02:19.007131639 +0000 UTC m=+0.148657454 container create d9f2e2f343bd2864acaedceed80b05e8d9a32010c49e94030d27956c50b2a4d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1-deactivate, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-14T18:02:19.518 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:19 smithi178 podman[153560]: 2024-02-14 18:02:19.226020691 +0000 UTC m=+0.367546508 container init d9f2e2f343bd2864acaedceed80b05e8d9a32010c49e94030d27956c50b2a4d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-02-14T18:02:19.518 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:19 smithi178 podman[153560]: 2024-02-14 18:02:19.23185133 +0000 UTC m=+0.373377151 container start d9f2e2f343bd2864acaedceed80b05e8d9a32010c49e94030d27956c50b2a4d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-14T18:02:19.518 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:19 smithi178 podman[153560]: 2024-02-14 18:02:19.34272547 +0000 UTC m=+0.484251317 container attach d9f2e2f343bd2864acaedceed80b05e8d9a32010c49e94030d27956c50b2a4d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-14T18:02:19.796 INFO:journalctl@ceph.osd.1.smithi178.stdout:Feb 14 18:02:19 smithi178 podman[153560]: 2024-02-14 18:02:19.517476929 +0000 UTC m=+0.659002735 container died d9f2e2f343bd2864acaedceed80b05e8d9a32010c49e94030d27956c50b2a4d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:02:20.085 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.1.service' 2024-02-14T18:02:20.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:02:20.127 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-14T18:02:20.127 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-14T18:02:20.127 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.2 2024-02-14T18:02:20.332 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:20 smithi178 systemd[1]: Stopping Ceph osd.2 for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:02:20.832 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:20 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2[56763]: 2024-02-14T18:02:20.510+0000 7f0a4e650700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:02:20.832 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:20 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2[56763]: 2024-02-14T18:02:20.510+0000 7f0a4e650700 -1 osd.2 785 *** Got signal Terminated *** 2024-02-14T18:02:20.833 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:20 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2[56763]: 2024-02-14T18:02:20.510+0000 7f0a4e650700 -1 osd.2 785 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:02:23.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:23 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:23.570Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-14T18:02:23.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:23 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:23.571Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused" 2024-02-14T18:02:23.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:23 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:23.572Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-14T18:02:26.317 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:26 smithi178 podman[153691]: 2024-02-14 18:02:26.02472631 +0000 UTC m=+5.639773506 container died 439d0e6c17bb95c8b7be126dc0c5ed174aecc792ed22f8dc2976411b5c6b3ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-02-14T18:02:26.582 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:26 smithi178 podman[153691]: 2024-02-14 18:02:26.423552778 +0000 UTC m=+6.038599974 container cleanup 439d0e6c17bb95c8b7be126dc0c5ed174aecc792ed22f8dc2976411b5c6b3ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T18:02:26.583 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:26 smithi178 bash[153691]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2 2024-02-14T18:02:26.996 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:26 smithi178 podman[153724]: 2024-02-14 18:02:26.708240866 +0000 UTC m=+0.678657491 container remove 439d0e6c17bb95c8b7be126dc0c5ed174aecc792ed22f8dc2976411b5c6b3ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , ceph=True) 2024-02-14T18:02:27.303 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:26 smithi178 podman[153818]: 2024-02-14 18:02:26.897946125 +0000 UTC m=+0.017093090 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:02:27.303 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:27 smithi178 podman[153818]: 2024-02-14 18:02:27.000641568 +0000 UTC m=+0.119788594 container create d07cc7a14d186a1c123457e242f9d597b056d3c4b8a5b31066a1cc2df66fdc20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:02:27.303 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:27 smithi178 podman[153818]: 2024-02-14 18:02:27.211261299 +0000 UTC m=+0.330408248 container init d07cc7a14d186a1c123457e242f9d597b056d3c4b8a5b31066a1cc2df66fdc20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-14T18:02:27.303 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:27 smithi178 podman[153818]: 2024-02-14 18:02:27.216737605 +0000 UTC m=+0.335884556 container start d07cc7a14d186a1c123457e242f9d597b056d3c4b8a5b31066a1cc2df66fdc20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T18:02:27.303 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:27 smithi178 podman[153818]: 2024-02-14 18:02:27.302583751 +0000 UTC m=+0.421730702 container attach d07cc7a14d186a1c123457e242f9d597b056d3c4b8a5b31066a1cc2df66fdc20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-14T18:02:27.582 INFO:journalctl@ceph.osd.2.smithi178.stdout:Feb 14 18:02:27 smithi178 podman[153818]: 2024-02-14 18:02:27.501050965 +0000 UTC m=+0.620197919 container died d07cc7a14d186a1c123457e242f9d597b056d3c4b8a5b31066a1cc2df66fdc20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-14T18:02:28.083 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.2.service' 2024-02-14T18:02:28.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:02:28.123 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-14T18:02:28.123 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-14T18:02:28.123 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.3 2024-02-14T18:02:28.332 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:28 smithi178 systemd[1]: Stopping Ceph osd.3 for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:02:28.832 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:28 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3[63393]: 2024-02-14T18:02:28.452+0000 7f74053ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:02:28.832 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:28 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3[63393]: 2024-02-14T18:02:28.452+0000 7f74053ea700 -1 osd.3 785 *** Got signal Terminated *** 2024-02-14T18:02:28.832 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:28 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3[63393]: 2024-02-14T18:02:28.452+0000 7f74053ea700 -1 osd.3 785 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:02:31.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:30 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:30.806+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:31.219 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:30 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:30.837+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:31.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:30 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:30.921+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:32.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:31.799+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:32.219 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:31.819+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:32.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:31 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:31.901+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:32.832 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:32 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3[63393]: 2024-02-14T18:02:32.417+0000 7f73fd9d6700 -1 osd.3 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.079131+0000 front 2024-02-14T18:02:06.079262+0000 (oldest deadline 2024-02-14T18:02:31.979219+0000) 2024-02-14T18:02:33.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:32 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:32.915+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:33.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:32 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:32.840+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:33.219 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:32 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:32.785+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:33.718 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:33 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:33.300+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:08.570768+0000 front 2024-02-14T18:02:08.570942+0000 (oldest deadline 2024-02-14T18:02:33.270883+0000) 2024-02-14T18:02:33.832 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:33 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:33.572Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused" 2024-02-14T18:02:33.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:33 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:33.572Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused" 2024-02-14T18:02:33.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:33 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:33.572Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-14T18:02:34.160 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:33 smithi178 podman[153947]: 2024-02-14 18:02:33.882606565 +0000 UTC m=+5.537670340 container died a4a3f9f44cfca534130e0892cce90106349dc8c4a0e7c15fa79f9dc49b855b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3, org.label-schema.build-date=20240214, ceph=True, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-14T18:02:34.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:33 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:33.926+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:34.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:33 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:33.813+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:34.219 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:33 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:33.782+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:34.510 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:34 smithi178 podman[153947]: 2024-02-14 18:02:34.249843964 +0000 UTC m=+5.904907740 container cleanup a4a3f9f44cfca534130e0892cce90106349dc8c4a0e7c15fa79f9dc49b855b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-02-14T18:02:34.510 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:34 smithi178 bash[153947]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3 2024-02-14T18:02:34.718 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:34 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:34.265+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:08.570768+0000 front 2024-02-14T18:02:08.570942+0000 (oldest deadline 2024-02-14T18:02:33.270883+0000) 2024-02-14T18:02:34.832 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:34 smithi178 podman[153970]: 2024-02-14 18:02:34.50930764 +0000 UTC m=+0.624108646 container remove a4a3f9f44cfca534130e0892cce90106349dc8c4a0e7c15fa79f9dc49b855b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-14T18:02:35.171 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:34 smithi178 podman[154069]: 2024-02-14 18:02:34.860046061 +0000 UTC m=+0.112396889 container create 5c6ab38ab94b11a55445333b555a0f3e31dbc84591230e491693b4c4e98f7b25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_BRANCH=HEAD) 2024-02-14T18:02:35.171 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:34 smithi178 podman[154069]: 2024-02-14 18:02:34.765777381 +0000 UTC m=+0.018128214 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:02:35.171 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:35 smithi178 podman[154069]: 2024-02-14 18:02:35.079106549 +0000 UTC m=+0.331457378 container init 5c6ab38ab94b11a55445333b555a0f3e31dbc84591230e491693b4c4e98f7b25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:02:35.171 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:35 smithi178 podman[154069]: 2024-02-14 18:02:35.084286093 +0000 UTC m=+0.336636922 container start 5c6ab38ab94b11a55445333b555a0f3e31dbc84591230e491693b4c4e98f7b25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T18:02:35.172 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:35 smithi178 podman[154069]: 2024-02-14 18:02:35.170386728 +0000 UTC m=+0.422737568 container attach 5c6ab38ab94b11a55445333b555a0f3e31dbc84591230e491693b4c4e98f7b25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-14T18:02:35.219 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:34 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:34.807+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:35.377 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:34 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:34.879+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:35.377 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:34 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:34.800+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:35.582 INFO:journalctl@ceph.osd.3.smithi178.stdout:Feb 14 18:02:35 smithi178 podman[154069]: 2024-02-14 18:02:35.373397888 +0000 UTC m=+0.625748711 container died 5c6ab38ab94b11a55445333b555a0f3e31dbc84591230e491693b4c4e98f7b25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-3-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True) 2024-02-14T18:02:35.718 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:35 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:35.299+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:08.570768+0000 front 2024-02-14T18:02:08.570942+0000 (oldest deadline 2024-02-14T18:02:33.270883+0000) 2024-02-14T18:02:35.965 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.3.service' 2024-02-14T18:02:36.005 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:02:36.005 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-14T18:02:36.005 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-14T18:02:36.005 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.4 2024-02-14T18:02:36.035 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:35 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:35.892+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:36.036 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:35 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:35.781+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:36.036 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:35 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:35.822+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:36.343 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:36 smithi184 systemd[1]: Stopping Ceph osd.4 for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:02:36.343 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:36.253+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:08.570768+0000 front 2024-02-14T18:02:08.570942+0000 (oldest deadline 2024-02-14T18:02:33.270883+0000) 2024-02-14T18:02:36.718 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:36.342+0000 7f65d9bcf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:02:36.719 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:36.342+0000 7f65d9bcf700 -1 osd.4 785 *** Got signal Terminated *** 2024-02-14T18:02:36.719 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:36.342+0000 7f65d9bcf700 -1 osd.4 785 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:02:37.218 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:36.872+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:37.219 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:36.865+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:37.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:36 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:36.763+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:37.718 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:37 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:37.226+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:08.570768+0000 front 2024-02-14T18:02:08.570942+0000 (oldest deadline 2024-02-14T18:02:33.270883+0000) 2024-02-14T18:02:38.219 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:37 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:37.817+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:38.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:37 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:37.844+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:38.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:37 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:37.751+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:38.718 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:38 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:38.245+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:08.570768+0000 front 2024-02-14T18:02:08.570942+0000 (oldest deadline 2024-02-14T18:02:33.270883+0000) 2024-02-14T18:02:38.969 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:38 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:38.860+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:38.969 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:38 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:38.860+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:38.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:38 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:38.716+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:38.969 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:38 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:38.796+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:39.716 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:39.286+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:08.570768+0000 front 2024-02-14T18:02:08.570942+0000 (oldest deadline 2024-02-14T18:02:33.270883+0000) 2024-02-14T18:02:39.716 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:39.286+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.772099+0000 front 2024-02-14T18:02:16.771992+0000 (oldest deadline 2024-02-14T18:02:38.972359+0000) 2024-02-14T18:02:39.968 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:39.715+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:39.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:39.715+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:39.969 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:39.892+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:39.969 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:39.892+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:39.969 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:39.785+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:39.969 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:39 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:39.785+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.656208+0000 front 2024-02-14T18:02:16.656226+0000 (oldest deadline 2024-02-14T18:02:39.456581+0000) 2024-02-14T18:02:40.718 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:40 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:40.237+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:08.570768+0000 front 2024-02-14T18:02:08.570942+0000 (oldest deadline 2024-02-14T18:02:33.270883+0000) 2024-02-14T18:02:40.719 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:40 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:40.237+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.772099+0000 front 2024-02-14T18:02:16.771992+0000 (oldest deadline 2024-02-14T18:02:38.972359+0000) 2024-02-14T18:02:41.212 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:40 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:40.914+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:41.213 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:40 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:40.914+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:41.213 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:40 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:40.780+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:41.213 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:40 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:40.780+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.656208+0000 front 2024-02-14T18:02:16.656226+0000 (oldest deadline 2024-02-14T18:02:39.456581+0000) 2024-02-14T18:02:41.213 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:40 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:40.757+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:41.213 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:40 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:40.757+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:41.469 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:41.211+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:08.570768+0000 front 2024-02-14T18:02:08.570942+0000 (oldest deadline 2024-02-14T18:02:33.270883+0000) 2024-02-14T18:02:41.469 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4[39020]: 2024-02-14T18:02:41.211+0000 7f65d21bb700 -1 osd.4 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.772099+0000 front 2024-02-14T18:02:16.771992+0000 (oldest deadline 2024-02-14T18:02:38.972359+0000) 2024-02-14T18:02:42.182 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:41 smithi184 podman[72922]: 2024-02-14 18:02:41.894598049 +0000 UTC m=+5.696322044 container died 3a610cc398382fb222d12e286bb52d80ba22636d82aa247ec1f58d91f8a37e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-14T18:02:42.182 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:41.937+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:42.182 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:41.937+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:42.182 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:41.777+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:42.183 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:41.777+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:42.183 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:41.799+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:42.183 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:41 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:41.799+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.656208+0000 front 2024-02-14T18:02:16.656226+0000 (oldest deadline 2024-02-14T18:02:39.456581+0000) 2024-02-14T18:02:42.469 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:42 smithi184 podman[72922]: 2024-02-14 18:02:42.297589567 +0000 UTC m=+6.099313561 container cleanup 3a610cc398382fb222d12e286bb52d80ba22636d82aa247ec1f58d91f8a37e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T18:02:42.469 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:42 smithi184 bash[72922]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4 2024-02-14T18:02:42.799 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:42 smithi184 podman[72945]: 2024-02-14 18:02:42.507142813 +0000 UTC m=+0.614545590 container remove 3a610cc398382fb222d12e286bb52d80ba22636d82aa247ec1f58d91f8a37e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-14T18:02:42.800 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:42 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:42.740+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:42.800 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:42 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:42.740+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:43.076 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:42 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:42.956+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:43.076 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:42 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:42.956+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:43.077 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:42 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:42.843+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:43.077 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:42 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:42.843+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.656208+0000 front 2024-02-14T18:02:16.656226+0000 (oldest deadline 2024-02-14T18:02:39.456581+0000) 2024-02-14T18:02:43.077 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:42 smithi184 podman[73041]: 2024-02-14 18:02:42.703084402 +0000 UTC m=+0.018408630 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:02:43.077 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:42 smithi184 podman[73041]: 2024-02-14 18:02:42.807944146 +0000 UTC m=+0.123268358 container create 8788fb358dc1699f5a4fe5a7cf200d8c6477d463ca81f7cb16fc5e0a73674bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-14T18:02:43.077 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:42 smithi184 podman[73041]: 2024-02-14 18:02:42.993663022 +0000 UTC m=+0.308987235 container init 8788fb358dc1699f5a4fe5a7cf200d8c6477d463ca81f7cb16fc5e0a73674bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T18:02:43.077 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:42 smithi184 podman[73041]: 2024-02-14 18:02:42.999390203 +0000 UTC m=+0.314714420 container start 8788fb358dc1699f5a4fe5a7cf200d8c6477d463ca81f7cb16fc5e0a73674bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-14T18:02:43.468 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:43 smithi184 podman[73041]: 2024-02-14 18:02:43.076742379 +0000 UTC m=+0.392066594 container attach 8788fb358dc1699f5a4fe5a7cf200d8c6477d463ca81f7cb16fc5e0a73674bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-02-14T18:02:43.469 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 14 18:02:43 smithi184 podman[73041]: 2024-02-14 18:02:43.292973873 +0000 UTC m=+0.608298095 container died 8788fb358dc1699f5a4fe5a7cf200d8c6477d463ca81f7cb16fc5e0a73674bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-4-deactivate, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-14T18:02:43.832 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:43 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:43.573Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-14T18:02:43.832 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:43 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:43.574Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused" 2024-02-14T18:02:43.832 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:43 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:43.574Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-14T18:02:43.930 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:43.815+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:43.930 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:43.815+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.656208+0000 front 2024-02-14T18:02:16.656226+0000 (oldest deadline 2024-02-14T18:02:39.456581+0000) 2024-02-14T18:02:43.931 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:43.695+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:43.931 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:43.695+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:43.953 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.4.service' 2024-02-14T18:02:44.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:02:44.037 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-14T18:02:44.037 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-14T18:02:44.037 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.5 2024-02-14T18:02:44.219 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:44 smithi184 systemd[1]: Stopping Ceph osd.5 for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:02:44.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:43.972+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:44.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:43 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:43.972+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:44.705 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:44.402+0000 7f16c5bbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:02:44.706 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:44.402+0000 7f16c5bbf700 -1 osd.5 785 *** Got signal Terminated *** 2024-02-14T18:02:44.706 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:44.402+0000 7f16c5bbf700 -1 osd.5 785 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:02:44.969 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:44.846+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:44.969 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:44.846+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.656208+0000 front 2024-02-14T18:02:16.656226+0000 (oldest deadline 2024-02-14T18:02:39.456581+0000) 2024-02-14T18:02:44.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:44.704+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:44.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:44.704+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:45.469 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:44.994+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:45.469 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:44 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:44.994+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:46.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:45 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:45.975+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:46.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:45 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:45.975+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:46.219 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:45 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:45.854+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:46.219 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:45 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:45.854+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.656208+0000 front 2024-02-14T18:02:16.656226+0000 (oldest deadline 2024-02-14T18:02:39.456581+0000) 2024-02-14T18:02:46.220 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:45 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:45.750+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:46.220 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:45 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:45.750+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:46.989 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:46 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:46.736+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:46.989 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:46 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:46.736+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:46.990 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:46 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:46.820+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:46.990 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:46 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:46.820+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.656208+0000 front 2024-02-14T18:02:16.656226+0000 (oldest deadline 2024-02-14T18:02:39.456581+0000) 2024-02-14T18:02:46.990 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:46 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:46.820+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:21.157433+0000 front 2024-02-14T18:02:21.157402+0000 (oldest deadline 2024-02-14T18:02:46.457405+0000) 2024-02-14T18:02:47.468 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:46 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:46.987+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:47.469 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:46 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:46.987+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:47.966 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:47 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:47.688+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:47.966 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:47 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:47.688+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:47.966 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:47 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:47.833+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:47.967 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:47 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:47.833+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.656208+0000 front 2024-02-14T18:02:16.656226+0000 (oldest deadline 2024-02-14T18:02:39.456581+0000) 2024-02-14T18:02:47.967 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:47 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:47.833+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:21.157433+0000 front 2024-02-14T18:02:21.157402+0000 (oldest deadline 2024-02-14T18:02:46.457405+0000) 2024-02-14T18:02:48.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:47 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:47.965+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:48.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:47 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:47.965+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:48.951 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:48 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:48.675+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:48.951 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:48 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:48.675+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:48.951 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:48 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:48.867+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:07.354936+0000 front 2024-02-14T18:02:07.354872+0000 (oldest deadline 2024-02-14T18:02:30.254738+0000) 2024-02-14T18:02:48.951 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:48 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:48.867+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:16.656208+0000 front 2024-02-14T18:02:16.656226+0000 (oldest deadline 2024-02-14T18:02:39.456581+0000) 2024-02-14T18:02:48.951 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:48 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:48.867+0000 7f16c21b3700 -1 osd.5 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:21.157433+0000 front 2024-02-14T18:02:21.157402+0000 (oldest deadline 2024-02-14T18:02:46.457405+0000) 2024-02-14T18:02:48.952 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:48 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5[45378]: 2024-02-14T18:02:48.868+0000 7f16c21b3700 -1 osd.5 785 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.34367.0:4225 5.15 5:a93a5511:::notify.2:head [watch ping cookie 94510226532480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e785) 2024-02-14T18:02:49.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:48 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:48.950+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:49.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:48 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:48.950+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:49.958 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:49 smithi184 podman[73174]: 2024-02-14 18:02:49.941179218 +0000 UTC m=+5.666534757 container died f9791541620eb244cbf359028c263227ee9605ee9fe02cd53059c30eff30812d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T18:02:49.958 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:49 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:49.693+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:49.959 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:49 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:49.693+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:49.959 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:49 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:49.693+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:02:50.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:49 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:49.956+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:50.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:49 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:49.956+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:50.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:49 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:49.956+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.560671+0000 front 2024-02-14T18:02:24.560615+0000 (oldest deadline 2024-02-14T18:02:49.160941+0000) 2024-02-14T18:02:50.707 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:50 smithi184 podman[73174]: 2024-02-14 18:02:50.257883387 +0000 UTC m=+5.983238920 container cleanup f9791541620eb244cbf359028c263227ee9605ee9fe02cd53059c30eff30812d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T18:02:50.707 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:50 smithi184 bash[73174]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5 2024-02-14T18:02:50.707 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:50 smithi184 podman[73206]: 2024-02-14 18:02:50.459220359 +0000 UTC m=+0.517958658 container remove f9791541620eb244cbf359028c263227ee9605ee9fe02cd53059c30eff30812d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-02-14T18:02:50.969 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:50 smithi184 podman[73303]: 2024-02-14 18:02:50.768027886 +0000 UTC m=+0.018312421 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:02:50.969 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:50 smithi184 podman[73303]: 2024-02-14 18:02:50.876658598 +0000 UTC m=+0.126943120 container create d646882918a777c2179a8ccf7e67ceda16955992e65d8a1543caa1424568859b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T18:02:50.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:50 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:50.705+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:50.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:50 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:50.705+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:50.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:50 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:50.705+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:02:51.404 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:50 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:50.998+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:51.405 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:50 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:50.998+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:51.405 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:50 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:50.998+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.560671+0000 front 2024-02-14T18:02:24.560615+0000 (oldest deadline 2024-02-14T18:02:49.160941+0000) 2024-02-14T18:02:51.405 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:51 smithi184 podman[73303]: 2024-02-14 18:02:51.112323407 +0000 UTC m=+0.362607929 container init d646882918a777c2179a8ccf7e67ceda16955992e65d8a1543caa1424568859b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5-deactivate, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-14T18:02:51.405 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:51 smithi184 podman[73303]: 2024-02-14 18:02:51.11838249 +0000 UTC m=+0.368667012 container start d646882918a777c2179a8ccf7e67ceda16955992e65d8a1543caa1424568859b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5-deactivate, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-14T18:02:51.405 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:51 smithi184 podman[73303]: 2024-02-14 18:02:51.203744435 +0000 UTC m=+0.454028968 container attach d646882918a777c2179a8ccf7e67ceda16955992e65d8a1543caa1424568859b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux ) 2024-02-14T18:02:51.719 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:51 smithi184 podman[73303]: 2024-02-14 18:02:51.404658548 +0000 UTC m=+0.654943077 container died d646882918a777c2179a8ccf7e67ceda16955992e65d8a1543caa1424568859b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5) 2024-02-14T18:02:51.985 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 14 18:02:51 smithi184 podman[73350]: 2024-02-14 18:02:51.982281856 +0000 UTC m=+0.572162824 container remove d646882918a777c2179a8ccf7e67ceda16955992e65d8a1543caa1424568859b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T18:02:51.985 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:51.731+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:51.985 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:51.731+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:51.985 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:51 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:51.731+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:02:52.003 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.5.service' 2024-02-14T18:02:52.046 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:02:52.046 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-14T18:02:52.047 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-14T18:02:52.047 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.6 2024-02-14T18:02:52.388 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:52 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:52.031+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:52.388 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:52 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:52.031+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:52.389 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:52 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:52.031+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.560671+0000 front 2024-02-14T18:02:24.560615+0000 (oldest deadline 2024-02-14T18:02:49.160941+0000) 2024-02-14T18:02:52.389 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:52 smithi184 systemd[1]: Stopping Ceph osd.6 for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:02:52.389 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:52 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:52.386+0000 7fc2cd599700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:02:52.389 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:52 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:52.386+0000 7fc2cd599700 -1 osd.6 785 *** Got signal Terminated *** 2024-02-14T18:02:52.389 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:52 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:52.386+0000 7fc2cd599700 -1 osd.6 785 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:02:52.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:52 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:52.683+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:52.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:52 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:52.683+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:52.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:52 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:52.683+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:02:52.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:52 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:52.683+0000 7fd83bb79700 -1 osd.7 785 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24478.0:1209 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e785) 2024-02-14T18:02:53.469 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:53 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:53.028+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:53.469 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:53 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:53.028+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:53.469 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:53 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:53.028+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.560671+0000 front 2024-02-14T18:02:24.560615+0000 (oldest deadline 2024-02-14T18:02:49.160941+0000) 2024-02-14T18:02:53.832 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:53 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:53.573Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused" 2024-02-14T18:02:53.832 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:53 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:53.574Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused" 2024-02-14T18:02:53.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:02:53 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:02:53.574Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-14T18:02:53.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:53 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:53.687+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:53.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:53 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:53.687+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:53.969 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:53 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:53.687+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:02:54.468 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:54 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:54.023+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:54.469 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:54 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:54.023+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:54.469 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:54 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:54.023+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.560671+0000 front 2024-02-14T18:02:24.560615+0000 (oldest deadline 2024-02-14T18:02:49.160941+0000) 2024-02-14T18:02:54.992 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:54 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:54.732+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:54.992 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:54 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:54.732+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:54.992 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:54 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:54.732+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:02:54.992 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:54 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:54.732+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:29.363889+0000 front 2024-02-14T18:02:29.363919+0000 (oldest deadline 2024-02-14T18:02:54.063830+0000) 2024-02-14T18:02:55.305 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:54 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:54.991+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:55.306 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:54 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:54.991+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:55.306 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:54 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:54.991+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.560671+0000 front 2024-02-14T18:02:24.560615+0000 (oldest deadline 2024-02-14T18:02:49.160941+0000) 2024-02-14T18:02:55.306 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:54 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:54.991+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:32.662180+0000 front 2024-02-14T18:02:32.662155+0000 (oldest deadline 2024-02-14T18:02:54.962086+0000) 2024-02-14T18:02:56.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:55 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:55.961+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:56.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:55 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:55.961+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:56.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:55 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:55.961+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.560671+0000 front 2024-02-14T18:02:24.560615+0000 (oldest deadline 2024-02-14T18:02:49.160941+0000) 2024-02-14T18:02:56.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:55 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:55.961+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:32.662180+0000 front 2024-02-14T18:02:32.662155+0000 (oldest deadline 2024-02-14T18:02:54.962086+0000) 2024-02-14T18:02:56.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:55 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:55.767+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:56.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:55 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:55.767+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:56.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:55 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:55.767+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:02:56.220 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:55 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:55.767+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:29.363889+0000 front 2024-02-14T18:02:29.363919+0000 (oldest deadline 2024-02-14T18:02:54.063830+0000) 2024-02-14T18:02:57.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:57 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:57.005+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:09.057032+0000 front 2024-02-14T18:02:09.057157+0000 (oldest deadline 2024-02-14T18:02:30.057591+0000) 2024-02-14T18:02:57.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:57 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:57.005+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.758385+0000 front 2024-02-14T18:02:14.758514+0000 (oldest deadline 2024-02-14T18:02:38.758986+0000) 2024-02-14T18:02:57.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:57 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:57.005+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.560671+0000 front 2024-02-14T18:02:24.560615+0000 (oldest deadline 2024-02-14T18:02:49.160941+0000) 2024-02-14T18:02:57.219 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:57 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6[51611]: 2024-02-14T18:02:57.005+0000 7fc2c9b8d700 -1 osd.6 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:32.662180+0000 front 2024-02-14T18:02:32.662155+0000 (oldest deadline 2024-02-14T18:02:54.962086+0000) 2024-02-14T18:02:57.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:56 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:56.778+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:57.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:56 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:56.779+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:57.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:56 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:56.779+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:02:57.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:56 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:56.779+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:29.363889+0000 front 2024-02-14T18:02:29.363919+0000 (oldest deadline 2024-02-14T18:02:54.063830+0000) 2024-02-14T18:02:58.061 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:57 smithi184 podman[73434]: 2024-02-14 18:02:57.77579798 +0000 UTC m=+5.505913004 container died 97046e8bebcfd418eced4dca1f8d9df403719fd47137cc636ca8f345fe123988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux ) 2024-02-14T18:02:58.061 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:57 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:57.819+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:58.061 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:57 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:57.819+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:58.061 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:57 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:57.819+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:02:58.061 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:57 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:57.819+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:29.363889+0000 front 2024-02-14T18:02:29.363919+0000 (oldest deadline 2024-02-14T18:02:54.063830+0000) 2024-02-14T18:02:58.402 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:58 smithi184 podman[73434]: 2024-02-14 18:02:58.167846067 +0000 UTC m=+5.897961099 container cleanup 97046e8bebcfd418eced4dca1f8d9df403719fd47137cc636ca8f345fe123988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-14T18:02:58.402 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:58 smithi184 bash[73434]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6 2024-02-14T18:02:58.718 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:58 smithi184 podman[73457]: 2024-02-14 18:02:58.402419214 +0000 UTC m=+0.620636014 container remove 97046e8bebcfd418eced4dca1f8d9df403719fd47137cc636ca8f345fe123988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-14T18:02:59.031 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:58 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:58.782+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:59.031 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:58 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:58.782+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:59.031 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:58 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:58.782+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:02:59.031 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:58 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:58.782+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:29.363889+0000 front 2024-02-14T18:02:29.363919+0000 (oldest deadline 2024-02-14T18:02:54.063830+0000) 2024-02-14T18:02:59.031 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:58 smithi184 podman[73554]: 2024-02-14 18:02:58.637895696 +0000 UTC m=+0.018226513 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:02:59.031 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:58 smithi184 podman[73554]: 2024-02-14 18:02:58.744962936 +0000 UTC m=+0.125293743 container create 81f4e88764ae4d1fb4e989d8fdda08c1463153ae59b137009dff71ee60f9e4b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-14T18:02:59.031 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:58 smithi184 podman[73554]: 2024-02-14 18:02:58.947934914 +0000 UTC m=+0.328265731 container init 81f4e88764ae4d1fb4e989d8fdda08c1463153ae59b137009dff71ee60f9e4b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-14T18:02:59.031 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:58 smithi184 podman[73554]: 2024-02-14 18:02:58.953604334 +0000 UTC m=+0.333935142 container start 81f4e88764ae4d1fb4e989d8fdda08c1463153ae59b137009dff71ee60f9e4b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T18:02:59.031 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:59 smithi184 podman[73554]: 2024-02-14 18:02:59.030667397 +0000 UTC m=+0.410998229 container attach 81f4e88764ae4d1fb4e989d8fdda08c1463153ae59b137009dff71ee60f9e4b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-14T18:02:59.468 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:59 smithi184 podman[73554]: 2024-02-14 18:02:59.246115066 +0000 UTC m=+0.626445883 container died 81f4e88764ae4d1fb4e989d8fdda08c1463153ae59b137009dff71ee60f9e4b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-02-14T18:02:59.802 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 14 18:02:59 smithi184 podman[73598]: 2024-02-14 18:02:59.800646966 +0000 UTC m=+0.549404536 container remove 81f4e88764ae4d1fb4e989d8fdda08c1463153ae59b137009dff71ee60f9e4b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-14T18:02:59.803 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:59 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:59.736+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:02:59.803 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:59 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:59.736+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:02:59.803 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:59 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:59.736+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:02:59.803 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:59 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:02:59.736+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:29.363889+0000 front 2024-02-14T18:02:29.363919+0000 (oldest deadline 2024-02-14T18:02:54.063830+0000) 2024-02-14T18:02:59.818 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.6.service' 2024-02-14T18:02:59.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:02:59.859 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-14T18:02:59.859 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-14T18:02:59.859 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.7 2024-02-14T18:03:00.215 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:02:59 smithi184 systemd[1]: Stopping Ceph osd.7 for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:03:00.468 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:00.214+0000 7fd84358d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:03:00.469 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:00.214+0000 7fd84358d700 -1 osd.7 785 *** Got signal Terminated *** 2024-02-14T18:03:00.469 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:00.214+0000 7fd84358d700 -1 osd.7 785 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:03:01.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:00.763+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:03:01.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:00.763+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:03:01.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:00.763+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:03:01.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:00 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:00.763+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:29.363889+0000 front 2024-02-14T18:02:29.363919+0000 (oldest deadline 2024-02-14T18:02:54.063830+0000) 2024-02-14T18:03:02.218 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:01.769+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:03:02.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:01.769+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:03:02.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:01.769+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:03:02.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:01 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:01.769+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:29.363889+0000 front 2024-02-14T18:02:29.363919+0000 (oldest deadline 2024-02-14T18:02:54.063830+0000) 2024-02-14T18:03:03.218 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:02 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:02.789+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:03:03.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:02 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:02.789+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:03:03.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:02 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:02.789+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:03:03.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:02 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:02.789+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:29.363889+0000 front 2024-02-14T18:02:29.363919+0000 (oldest deadline 2024-02-14T18:02:54.063830+0000) 2024-02-14T18:03:03.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:02 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:02.789+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.184:6802 osd.4 since back 2024-02-14T18:02:41.065138+0000 front 2024-02-14T18:02:41.065221+0000 (oldest deadline 2024-02-14T18:03:02.165268+0000) 2024-02-14T18:03:03.832 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:03 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:03:03.574Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-14T18:03:03.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:03 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:03:03.574Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused" 2024-02-14T18:03:03.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:03 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:03:03.575Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-14T18:03:04.218 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:03 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:03.784+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:03:04.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:03 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:03.784+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:03:04.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:03 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:03.784+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:03:04.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:03 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:03.784+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:29.363889+0000 front 2024-02-14T18:02:29.363919+0000 (oldest deadline 2024-02-14T18:02:54.063830+0000) 2024-02-14T18:03:04.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:03 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:03.784+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.184:6802 osd.4 since back 2024-02-14T18:02:41.065138+0000 front 2024-02-14T18:02:41.065221+0000 (oldest deadline 2024-02-14T18:03:02.165268+0000) 2024-02-14T18:03:05.218 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:04 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:04.780+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6803 osd.0 since back 2024-02-14T18:02:06.259609+0000 front 2024-02-14T18:02:06.259674+0000 (oldest deadline 2024-02-14T18:02:30.260249+0000) 2024-02-14T18:03:05.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:04 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:04.780+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6807 osd.1 since back 2024-02-14T18:02:14.961175+0000 front 2024-02-14T18:02:14.961319+0000 (oldest deadline 2024-02-14T18:02:39.061236+0000) 2024-02-14T18:03:05.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:04 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:04.780+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6811 osd.2 since back 2024-02-14T18:02:24.762846+0000 front 2024-02-14T18:02:24.762923+0000 (oldest deadline 2024-02-14T18:02:48.862833+0000) 2024-02-14T18:03:05.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:04 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:04.780+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.178:6815 osd.3 since back 2024-02-14T18:02:29.363889+0000 front 2024-02-14T18:02:29.363919+0000 (oldest deadline 2024-02-14T18:02:54.063830+0000) 2024-02-14T18:03:05.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:04 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7[57834]: 2024-02-14T18:03:04.780+0000 7fd83bb79700 -1 osd.7 785 heartbeat_check: no reply from 172.21.15.184:6802 osd.4 since back 2024-02-14T18:02:41.065138+0000 front 2024-02-14T18:02:41.065221+0000 (oldest deadline 2024-02-14T18:03:02.165268+0000) 2024-02-14T18:03:05.954 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:05 smithi184 podman[73681]: 2024-02-14 18:03:05.666267226 +0000 UTC m=+5.577046931 container died 44bf45e7d8577932982d607edce68cb94f9fd92c8e5a5b4555150d7bda2d21e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-14T18:03:06.218 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:06 smithi184 podman[73681]: 2024-02-14 18:03:06.053107994 +0000 UTC m=+5.963887702 container cleanup 44bf45e7d8577932982d607edce68cb94f9fd92c8e5a5b4555150d7bda2d21e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-14T18:03:06.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:06 smithi184 bash[73681]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7 2024-02-14T18:03:06.619 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:06 smithi184 podman[73703]: 2024-02-14 18:03:06.320998535 +0000 UTC m=+0.654108552 container remove 44bf45e7d8577932982d607edce68cb94f9fd92c8e5a5b4555150d7bda2d21e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-14T18:03:06.898 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:06 smithi184 podman[73798]: 2024-02-14 18:03:06.523333854 +0000 UTC m=+0.018550990 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:03:06.899 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:06 smithi184 podman[73798]: 2024-02-14 18:03:06.621604457 +0000 UTC m=+0.116821585 container create 38e905e206a414648b54d4677345c7829c3ba55cf9faf04a41714ca25fa2203b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-14T18:03:06.899 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:06 smithi184 podman[73798]: 2024-02-14 18:03:06.815444248 +0000 UTC m=+0.310661375 container init 38e905e206a414648b54d4677345c7829c3ba55cf9faf04a41714ca25fa2203b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T18:03:06.899 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:06 smithi184 podman[73798]: 2024-02-14 18:03:06.820690073 +0000 UTC m=+0.315907203 container start 38e905e206a414648b54d4677345c7829c3ba55cf9faf04a41714ca25fa2203b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-14T18:03:06.899 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:06 smithi184 podman[73798]: 2024-02-14 18:03:06.898617823 +0000 UTC m=+0.393834960 container attach 38e905e206a414648b54d4677345c7829c3ba55cf9faf04a41714ca25fa2203b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-02-14T18:03:07.219 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 14 18:03:07 smithi184 podman[73798]: 2024-02-14 18:03:07.107629841 +0000 UTC m=+0.602846971 container died 38e905e206a414648b54d4677345c7829c3ba55cf9faf04a41714ca25fa2203b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-14T18:03:07.756 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@osd.7.service' 2024-02-14T18:03:07.796 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:03:07.797 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-14T18:03:07.797 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-14T18:03:07.797 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@rgw.foo.a 2024-02-14T18:03:08.083 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 18:03:07 smithi178 systemd[1]: Stopping Ceph rgw.foo.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:03:08.582 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 18:03:08 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-rgw-foo-a[68781]: 2024-02-14T18:03:08.116+0000 7fca016d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:03:08.582 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 18:03:08 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-rgw-foo-a[68781]: 2024-02-14T18:03:08.116+0000 7fca0ff42ac0 -1 shutting down 2024-02-14T18:03:13.219 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:12.942Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.178:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.178:8765: connect: connection refused" 2024-02-14T18:03:13.219 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:12.943Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.178:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.178:8765: connect: connection refused" 2024-02-14T18:03:13.219 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:12.943Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.178:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.178:8765: connect: connection refused" 2024-02-14T18:03:13.219 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:12 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:12.944Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.178:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.178:8765: connect: connection refused" 2024-02-14T18:03:13.832 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:03:13.575Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused" 2024-02-14T18:03:13.832 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:03:13.575Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi178.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.178:8443: connect: connection refused" 2024-02-14T18:03:13.833 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:13 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:03:13.575Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-14T18:03:18.440 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 18:03:18 smithi178 bash[154210]: time="2024-02-14T18:03:18Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-14T18:03:18.440 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 18:03:18 smithi178 podman[154210]: 2024-02-14 18:03:18.144073397 +0000 UTC m=+10.142056097 container died a7caf9b666504047af6a0c1298f5e995210aadce4abd2df592a3e4dbd7f4171b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-rgw-foo-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-14T18:03:18.780 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 18:03:18 smithi178 podman[154210]: 2024-02-14 18:03:18.503862227 +0000 UTC m=+10.501844882 container cleanup a7caf9b666504047af6a0c1298f5e995210aadce4abd2df592a3e4dbd7f4171b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True) 2024-02-14T18:03:18.780 INFO:journalctl@ceph.rgw.foo.a.smithi178.stdout:Feb 14 18:03:18 smithi178 bash[154210]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-rgw-foo-a 2024-02-14T18:03:18.929 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@rgw.foo.a.service' 2024-02-14T18:03:18.967 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:03:18.967 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-14T18:03:18.967 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-14T18:03:18.967 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@prometheus.a 2024-02-14T18:03:19.256 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 systemd[1]: Stopping Ceph prometheus.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:03:19.538 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.256Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-14T18:03:19.538 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.256Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-14T18:03:19.538 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.256Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-14T18:03:19.538 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.256Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-14T18:03:19.538 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.256Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-14T18:03:19.538 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.256Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-14T18:03:19.538 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.256Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-14T18:03:19.538 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.256Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-14T18:03:19.538 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.256Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-14T18:03:19.538 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.257Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-14T18:03:19.538 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.257Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-14T18:03:19.539 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a[68228]: ts=2024-02-14T18:03:19.257Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-14T18:03:19.539 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 podman[73939]: 2024-02-14 18:03:19.266871927 +0000 UTC m=+0.102929465 container died a00d25be6bcb9a0e778ef83ead3d5f3d552c30d10a678543370a58f4037707dc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T18:03:19.888 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 podman[73939]: 2024-02-14 18:03:19.628341594 +0000 UTC m=+0.464399130 container cleanup a00d25be6bcb9a0e778ef83ead3d5f3d552c30d10a678543370a58f4037707dc (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T18:03:19.888 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 14 18:03:19 smithi184 bash[73939]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-prometheus-a 2024-02-14T18:03:20.044 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@prometheus.a.service' 2024-02-14T18:03:20.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:03:20.083 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-14T18:03:20.083 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm rm-cluster --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-14T18:03:20.269 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T18:03:22.082 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:21 smithi178 systemd[1]: Stopping Ceph alertmanager.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:03:22.417 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:22 smithi178 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a[79797]: ts=2024-02-14T18:03:22.119Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-14T18:03:22.417 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:22 smithi178 podman[154718]: 2024-02-14 18:03:22.161023499 +0000 UTC m=+0.142264369 container died 68f4ed8d5cca2c4a42ccaab3df368d4637d4fb97653b1892651a11cf197b9db3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T18:03:22.417 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:22 smithi178 podman[154718]: 2024-02-14 18:03:22.416792499 +0000 UTC m=+0.398033376 container cleanup 68f4ed8d5cca2c4a42ccaab3df368d4637d4fb97653b1892651a11cf197b9db3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T18:03:22.685 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:22 smithi178 bash[154718]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a 2024-02-14T18:03:23.075 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:22 smithi178 podman[154737]: 2024-02-14 18:03:22.684711374 +0000 UTC m=+0.521598957 container remove 68f4ed8d5cca2c4a42ccaab3df368d4637d4fb97653b1892651a11cf197b9db3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T18:03:23.076 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:22 smithi178 podman[154737]: 2024-02-14 18:03:22.760273806 +0000 UTC m=+0.597161346 volume remove 7877abb90a56773027160f5a13f00a76de62feee3dbfe74676017e64054f9dd7 2024-02-14T18:03:23.076 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:22 smithi178 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-14T18:03:23.076 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:22 smithi178 systemd[1]: Stopped Ceph alertmanager.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:03:23.076 INFO:journalctl@ceph.alertmanager.a.smithi178.stdout:Feb 14 18:03:22 smithi178 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@alertmanager.a.service: Consumed 1.810s CPU time. 2024-02-14T18:03:23.332 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 18:03:23 smithi178 systemd[1]: Stopping Ceph node-exporter.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:03:23.332 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 18:03:23 smithi178 podman[154856]: 2024-02-14 18:03:23.318404248 +0000 UTC m=+0.106358737 container died 591f4445a1bdc52086d54c89c1f062e0b3b2340419c0e71c50034a21cff930f6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T18:03:23.948 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 18:03:23 smithi178 podman[154856]: 2024-02-14 18:03:23.712956476 +0000 UTC m=+0.500910974 container cleanup 591f4445a1bdc52086d54c89c1f062e0b3b2340419c0e71c50034a21cff930f6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T18:03:23.948 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 18:03:23 smithi178 bash[154856]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a 2024-02-14T18:03:24.216 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 18:03:23 smithi178 podman[154874]: 2024-02-14 18:03:23.947353946 +0000 UTC m=+0.633635308 container remove 591f4445a1bdc52086d54c89c1f062e0b3b2340419c0e71c50034a21cff930f6 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T18:03:24.216 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 18:03:23 smithi178 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-14T18:03:24.216 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 18:03:24 smithi178 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-14T18:03:24.216 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 18:03:24 smithi178 systemd[1]: Stopped Ceph node-exporter.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:03:24.216 INFO:journalctl@ceph.node-exporter.a.smithi178.stdout:Feb 14 18:03:24 smithi178 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@node-exporter.a.service: Consumed 3.654s CPU time. 2024-02-14T18:03:24.690 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm rm-cluster --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-14T18:03:24.875 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T18:03:26.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:03:26 smithi184 systemd[1]: Stopping Ceph iscsi.iscsi.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:03:26.469 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:03:26 smithi184 ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62948]: debug Shutdown received 2024-02-14T18:03:36.718 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:03:36 smithi184 bash[74362]: time="2024-02-14T18:03:36Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-02-14T18:03:36.719 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:03:36 smithi184 podman[74362]: 2024-02-14 18:03:36.383894874 +0000 UTC m=+10.127815605 container died d997ad155e7697b7ddb3c31d713b5aa1aeb6dda720e2ecc21f21b0f3b709a73c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-14T18:03:37.115 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:03:36 smithi184 podman[74362]: 2024-02-14 18:03:36.841967566 +0000 UTC m=+10.585888293 container cleanup d997ad155e7697b7ddb3c31d713b5aa1aeb6dda720e2ecc21f21b0f3b709a73c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T18:03:37.115 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:03:36 smithi184 bash[74362]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a 2024-02-14T18:03:37.451 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:03:37 smithi184 podman[74389]: 2024-02-14 18:03:37.115476249 +0000 UTC m=+0.730061606 container remove d997ad155e7697b7ddb3c31d713b5aa1aeb6dda720e2ecc21f21b0f3b709a73c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-14T18:03:37.451 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:03:37 smithi184 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-14T18:03:37.451 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:03:37 smithi184 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-14T18:03:37.451 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:03:37 smithi184 systemd[1]: Stopped Ceph iscsi.iscsi.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:03:37.451 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 14 18:03:37 smithi184 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@iscsi.iscsi.a.service: Consumed 3.440s CPU time. 2024-02-14T18:03:38.719 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 18:03:38 smithi184 systemd[1]: Stopping Ceph grafana.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:03:39.122 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 18:03:38 smithi184 podman[74666]: 2024-02-14 18:03:38.859953815 +0000 UTC m=+0.108299881 container died 03ec241e3c5ec5e2054df2e49810da0ca652b1f62b6859c11a2549ea32b3bd70 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a, io.openshift.tags=base rhel8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, release=239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, vendor=Red Hat, Inc., io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, description=Ceph Grafana Container, io.buildah.version=1.32.0, name=ubi8, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner ) 2024-02-14T18:03:39.381 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 18:03:39 smithi184 podman[74666]: 2024-02-14 18:03:39.196429821 +0000 UTC m=+0.444775886 container cleanup 03ec241e3c5ec5e2054df2e49810da0ca652b1f62b6859c11a2549ea32b3bd70 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., vcs-type=git, release=239.1651231664, name=ubi8, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, io.openshift.expose-services=, version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8) 2024-02-14T18:03:39.381 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 18:03:39 smithi184 bash[74666]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a 2024-02-14T18:03:39.634 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 18:03:39 smithi184 podman[74685]: 2024-02-14 18:03:39.381052412 +0000 UTC m=+0.523955050 container remove 03ec241e3c5ec5e2054df2e49810da0ca652b1f62b6859c11a2549ea32b3bd70 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-grafana-a, vendor=Red Hat, Inc., release=239.1651231664, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, architecture=x86_64, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, name=ubi8, com.redhat.component=ubi8-container, build-date=2022-04-29T11:48:11.055897, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, description=Ceph Grafana Container, io.buildah.version=1.32.0, distribution-scope=public) 2024-02-14T18:03:39.634 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 18:03:39 smithi184 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-14T18:03:39.634 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 18:03:39 smithi184 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-14T18:03:39.634 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 18:03:39 smithi184 systemd[1]: Stopped Ceph grafana.a for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:03:39.634 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 14 18:03:39 smithi184 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@grafana.a.service: Consumed 10.974s CPU time. 2024-02-14T18:03:39.968 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 18:03:39 smithi184 systemd[1]: Stopping Ceph node-exporter.b for 6e1588ec-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:03:40.342 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 18:03:40 smithi184 podman[74802]: 2024-02-14 18:03:40.055420354 +0000 UTC m=+0.095070147 container died daa702fa9f4132991e4628270d71c53c2fec07851d2b5ae315af8aa95fa940e2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T18:03:40.659 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 18:03:40 smithi184 podman[74802]: 2024-02-14 18:03:40.42488697 +0000 UTC m=+0.464536770 container cleanup daa702fa9f4132991e4628270d71c53c2fec07851d2b5ae315af8aa95fa940e2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T18:03:40.659 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 18:03:40 smithi184 bash[74802]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b 2024-02-14T18:03:40.925 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 18:03:40 smithi184 podman[74821]: 2024-02-14 18:03:40.659149664 +0000 UTC m=+0.608583010 container remove daa702fa9f4132991e4628270d71c53c2fec07851d2b5ae315af8aa95fa940e2 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T18:03:40.926 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 18:03:40 smithi184 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-14T18:03:40.926 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 18:03:40 smithi184 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-14T18:03:40.926 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 18:03:40 smithi184 systemd[1]: Stopped Ceph node-exporter.b for 6e1588ec-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:03:40.926 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 14 18:03:40 smithi184 systemd[1]: ceph-6e1588ec-cb5f-11ee-95b9-87774f69a715@node-exporter.b.service: Consumed 3.683s CPU time. 2024-02-14T18:03:41.406 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T18:03:41.443 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T18:03:41.472 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-14T18:03:41.474 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793/remote/smithi178/crash 2024-02-14T18:03:41.475 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/crash -- . 2024-02-14T18:03:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-14T18:03:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2024-02-14T18:03:41.533 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793/remote/smithi184/crash 2024-02-14T18:03:41.534 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/crash -- . 2024-02-14T18:03:41.571 INFO:teuthology.orchestra.run.smithi184.stderr:tar: /var/lib/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-14T18:03:41.572 INFO:teuthology.orchestra.run.smithi184.stderr:tar: Error is not recoverable: exiting now 2024-02-14T18:03:41.573 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-14T18:03:41.573 DEBUG:teuthology.orchestra.run.smithi178:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-14T18:03:41.614 INFO:teuthology.orchestra.run.smithi178.stdout:2024-02-14T17:50:00.000139+0000 mon.a (mon.0) 2414 : cluster 3 [WRN] CACHE_POOL_NEAR_FULL: 1 cache pools at or near target size 2024-02-14T18:03:41.614 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-14T18:03:41.614 DEBUG:teuthology.orchestra.run.smithi178:> sudo egrep '\[SEC\]' /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-14T18:03:41.686 DEBUG:teuthology.orchestra.run.smithi178:> sudo egrep '\[ERR\]' /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-14T18:03:41.758 DEBUG:teuthology.orchestra.run.smithi178:> sudo egrep '\[WRN\]' /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-14T18:03:41.828 INFO:teuthology.orchestra.run.smithi178.stdout:2024-02-14T17:50:00.000139+0000 mon.a (mon.0) 2414 : cluster 3 [WRN] CACHE_POOL_NEAR_FULL: 1 cache pools at or near target size 2024-02-14T18:03:41.828 INFO:tasks.cephadm:Compressing logs... 2024-02-14T18:03:41.829 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-14T18:03:41.870 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-14T18:03:41.894 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-14T18:03:41.894 INFO:teuthology.orchestra.run.smithi178.stderr:: No such file or directory 2024-02-14T18:03:41.895 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mon.a.log 2024-02-14T18:03:41.897 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-14T18:03:41.897 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-14T18:03:41.897 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.1.log 2024-02-14T18:03:41.897 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mon.a.log: /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.audit.log: /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.3.log 2024-02-14T18:03:41.897 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mgr.y.log 2024-02-14T18:03:41.897 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.0.log 2024-02-14T18:03:41.898 INFO:teuthology.orchestra.run.smithi184.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-02-14T18:03:41.898 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-02-14T18:03:41.898 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.6.log 2024-02-14T18:03:41.898 INFO:teuthology.orchestra.run.smithi184.stderr:gzip/var/log/ceph/cephadm.log: -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-14T18:03:41.899 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-14T18:03:41.900 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.4.log 2024-02-14T18:03:41.900 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.7.log 2024-02-14T18:03:41.900 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.5.log 2024-02-14T18:03:41.904 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.4.log: /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mgr.x.log 2024-02-14T18:03:41.904 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/tcmu-runner.log 2024-02-14T18:03:41.905 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-14T18:03:41.905 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/tcmu-runner.log: 68.0% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/tcmu-runner.log.gz 2024-02-14T18:03:41.905 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.log 2024-02-14T18:03:41.905 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log: 81.7% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-14T18:03:41.905 INFO:teuthology.orchestra.run.smithi184.stderr: 85.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-14T18:03:41.905 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-14T18:03:41.905 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mon.b.log 2024-02-14T18:03:41.906 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.0.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-14T18:03:41.908 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-volume.log: 86.9% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-14T18:03:41.910 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.log 2024-02-14T18:03:41.910 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log: 89.0% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-14T18:03:41.911 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-14T18:03:41.911 INFO:teuthology.orchestra.run.smithi184.stderr: 91.7% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mgr.x.log.gz 2024-02-14T18:03:41.915 INFO:teuthology.orchestra.run.smithi178.stderr: 95.2% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-14T18:03:41.916 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.2.log 2024-02-14T18:03:41.916 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-14T18:03:41.917 INFO:teuthology.orchestra.run.smithi184.stderr: 90.7% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-14T18:03:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.2.log: 92.7% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-14T18:03:41.930 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mon.c.log 2024-02-14T18:03:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-volume.log: 93.0% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-14T18:03:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mon.c.log: 90.8% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-14T18:03:42.301 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mgr.y.log.gz 2024-02-14T18:03:43.029 INFO:teuthology.orchestra.run.smithi184.stderr: 92.6% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mon.b.log.gz 2024-02-14T18:03:43.479 INFO:teuthology.orchestra.run.smithi178.stderr: 92.0% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mon.c.log.gz 2024-02-14T18:03:44.428 INFO:teuthology.orchestra.run.smithi178.stderr: 91.7% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-mon.a.log.gz 2024-02-14T18:03:48.256 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.6.log.gz 2024-02-14T18:03:48.657 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.5.log.gz 2024-02-14T18:03:48.785 INFO:teuthology.orchestra.run.smithi178.stderr: 93.8% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.1.log.gz 2024-02-14T18:03:48.791 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.4.log.gz 2024-02-14T18:03:48.888 INFO:teuthology.orchestra.run.smithi178.stderr: 93.8% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.2.log.gz 2024-02-14T18:03:49.055 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.7.log.gz 2024-02-14T18:03:49.057 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-02-14T18:03:49.057 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m7.172s 2024-02-14T18:03:49.057 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m27.346s 2024-02-14T18:03:49.057 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m0.961s 2024-02-14T18:03:49.149 INFO:teuthology.orchestra.run.smithi178.stderr: 93.9% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.3.log.gz 2024-02-14T18:03:49.245 INFO:teuthology.orchestra.run.smithi178.stderr: 93.9% -- replaced with /var/log/ceph/6e1588ec-cb5f-11ee-95b9-87774f69a715/ceph-osd.0.log.gz 2024-02-14T18:03:49.247 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-02-14T18:03:49.247 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m7.365s 2024-02-14T18:03:49.247 INFO:teuthology.orchestra.run.smithi178.stderr:user 0m31.839s 2024-02-14T18:03:49.247 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m1.149s 2024-02-14T18:03:49.247 INFO:tasks.cephadm:Archiving logs... 2024-02-14T18:03:49.248 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793/remote/smithi178/log 2024-02-14T18:03:49.249 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-14T18:03:53.456 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793/remote/smithi184/log 2024-02-14T18:03:53.457 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-14T18:03:57.237 INFO:tasks.cephadm:Removing cluster... 2024-02-14T18:03:57.238 DEBUG:teuthology.orchestra.run.smithi178:> sudo cephadm rm-cluster --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 --force 2024-02-14T18:03:57.428 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T18:03:57.714 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm rm-cluster --fsid 6e1588ec-cb5f-11ee-95b9-87774f69a715 --force 2024-02-14T18:03:57.906 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 6e1588ec-cb5f-11ee-95b9-87774f69a715 2024-02-14T18:03:58.203 INFO:tasks.cephadm:Teardown complete 2024-02-14T18:03:58.204 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-14T18:03:58.214 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-14T18:03:58.214 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-14T18:03:58.217 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-14T18:03:58.294 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-14T18:03:58.303 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-14T18:03:58.304 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T18:03:58.307 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T18:03:58.318 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-02-14T18:03:58.321 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-02-14T18:03:58.322 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T18:03:58.322 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-02-14T18:03:58.322 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 5 +2351us[+2349us] +/- 113ms 2024-02-14T18:03:58.322 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv02.front.sepia.ceph.com 3 6 377 5 -559us[ -561us] +/- 62ms 2024-02-14T18:03:58.322 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -805us[ -805us] +/- 81ms 2024-02-14T18:03:58.322 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T18:03:58.325 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T18:03:58.325 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-02-14T18:03:58.325 INFO:teuthology.orchestra.run.smithi184.stdout:^- hv01.front.sepia.ceph.com 2 6 377 1 +3010us[+3010us] +/- 113ms 2024-02-14T18:03:58.325 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv02.front.sepia.ceph.com 3 6 377 5 +118us[ +127us] +/- 62ms 2024-02-14T18:03:58.326 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 -148us[ -148us] +/- 81ms 2024-02-14T18:03:58.326 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T18:03:58.327 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-14T18:03:58.337 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-14T18:03:58.338 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-14T18:03:58.346 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-14T18:03:58.443 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-14T18:03:58.517 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T18:03:58.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T18:03:58.548 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T18:03:58.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T18:03:58.578 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-14T18:03:58.587 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-14T18:03:58.595 INFO:teuthology.task.internal:Duration was 2142.969635 seconds 2024-02-14T18:03:58.596 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-14T18:03:58.604 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-14T18:03:58.604 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-14T18:03:58.606 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-14T18:03:58.646 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T18:03:58.662 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T18:03:58.873 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-14T18:03:58.873 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-02-14T18:03:58.873 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-14T18:03:58.932 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-02-14T18:03:58.933 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-14T18:03:58.954 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-14T18:03:58.954 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-14T18:03:58.975 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-14T18:03:59.023 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-14T18:03:59.023 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-14T18:03:59.043 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-14T18:03:59.447 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-14T18:03:59.457 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-14T18:03:59.458 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-14T18:03:59.484 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-14T18:03:59.511 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-14T18:03:59.553 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-14T18:03:59.556 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-14T18:03:59.579 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-02-14T18:03:59.581 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-02-14T18:03:59.603 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-14T18:03:59.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T18:03:59.649 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-14T18:03:59.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T18:03:59.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-14T18:03:59.673 INFO:teuthology.task.internal:Transferring archived files... 2024-02-14T18:03:59.674 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793/remote/smithi178 2024-02-14T18:03:59.674 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-14T18:03:59.750 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793/remote/smithi184 2024-02-14T18:03:59.750 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-14T18:03:59.788 INFO:teuthology.task.internal:Removing archive directory... 2024-02-14T18:03:59.788 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-14T18:03:59.790 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-14T18:03:59.837 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-14T18:03:59.849 INFO:teuthology.task.internal:Not uploading archives. 2024-02-14T18:03:59.850 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-14T18:03:59.859 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-14T18:03:59.859 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-14T18:03:59.861 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-14T18:03:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: 265293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 14 18:03 /home/ubuntu/cephtest 2024-02-14T18:03:59.891 INFO:teuthology.orchestra.run.smithi184.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 14 18:03 /home/ubuntu/cephtest 2024-02-14T18:03:59.892 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-14T18:03:59.902 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-14T18:03:59.922 INFO:teuthology.nuke:Checking targets against current locks 2024-02-14T18:03:59.952 DEBUG:teuthology.nuke:shortname: smithi178 2024-02-14T18:03:59.952 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T18:03:59.972 DEBUG:teuthology.nuke:shortname: smithi184 2024-02-14T18:03:59.972 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T18:03:59.975 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 17:19:19.975016', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMsvxKdQsDa8qMY7n8wMF12iX6/2AeZvMJA+NH0/z+tBOwWZPHhgYwtuPScajjHIMpOmxa7cWAqHdYjT3yJ+Me0='} 2024-02-14T18:03:59.998 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559793', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 17:19:19.976716', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCWf6xgbepP4XQoSuLV9ftXptjCbEYXLJNvnDl37roFCrxZjL69Fc0q1OiLtzlR9cNyA6aDkgHYTXjOPx4mymw='} 2024-02-14T18:04:00.025 INFO:teuthology.orchestra.console.smithi178:Power off 2024-02-14T18:04:00.026 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T18:04:00.040 INFO:teuthology.orchestra.console.smithi184:Power off 2024-02-14T18:04:00.040 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T18:04:00.050 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2024-02-14T18:04:00.051 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:04:00.070 DEBUG:teuthology.orchestra.console.smithi184:power off output: Chassis Power Control: Down/Off 2024-02-14T18:04:00.070 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:04:00.079 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2024-02-14T18:04:00.092 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2024-02-14T18:04:04.080 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:04:04.093 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:04:04.191 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2024-02-14T18:04:04.202 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2024-02-14T18:04:08.196 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:04:08.204 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:04:08.306 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2024-02-14T18:04:08.314 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off 2024-02-14T18:04:08.407 INFO:teuthology.orchestra.console.smithi178:Power off completed 2024-02-14T18:04:08.415 INFO:teuthology.orchestra.console.smithi184:Power off completed 2024-02-14T18:04:08.519 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} duration: 2142.9696350097656 failure_reason: '"2024-02-14T17:50:00.000139+0000 mon.a (mon.0) 2414 : cluster 3 [WRN] CACHE_POOL_NEAR_FULL: 1 cache pools at or near target size" in cluster log' flavor: default owner: scheduled_gabrioux@teuthology success: false 2024-02-14T18:04:08.519 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T18:04:08.600 INFO:teuthology.run:FAIL